OpenRaiser/Intern-Atlas huggingface dataset:
Control Improvisation with Probabilistic Temporal Specifications
Venue: arXiv cs.SY 2016
Year: 2016

We consider the problem of generating randomized control sequences for complex networked systems typically actuated by human agents. Our approach leverages a concept known as control improvisation, which is based on a combination of data-driven learning and controller synthesis from formal specifications. We learn from existing data a generative model (for instance, an explicit-duration hidden Markov model, or EDHMM) and then supervise this model in order to guarantee that the generated sequences satisfy some desirable specifications given in Probabilistic Computation Tree Logic (PCTL). We present an implementation of our approach and apply it to the problem of mimicking the use of lighting appliances in a residential unit, with potential applications to home security and resource management. We present experimental results showing that our approach produces realistic control sequences, similar to recorded data based on human actuation, while satisfying suitable formal requirements.
Information Extraction Under Privacy Constraints
Venue: arXiv cs.IT 2016
Year: 2016

A privacy-constrained information extraction problem is considered where for a pair of correlated discrete random variables $(X,Y)$ governed by a given joint distribution, an agent observes $Y$ and wants to convey to a potentially public user as much information about $Y$ as possible without compromising the amount of information revealed about $X$. To this end, the so-called {\em rate-privacy function} is introduced to quantify the maximal amount of information (measured in terms of mutual information) that can be extracted from $Y$ under a privacy constraint between $X$ and the extracted information, where privacy is measured using either mutual information or maximal correlation. Properties of the rate-privacy function are analyzed and information-theoretic and estimation-theoretic interpretations of it are presented for both the mutual information and maximal correlation privacy measures. It is also shown that the rate-privacy function admits a closed-form expression for a large family of joint distributions of $(X,Y)$. Finally, the rate-privacy function under the mutual information privacy measure is considered for the case where $(X,Y)$ has a joint probability density function by studying the problem where the extracted information is a uniform quantization of $Y$ corrupted by additive Gaussian noise. The asymptotic behavior of the rate-privacy function is studied as the quantization resolution grows without bound and it is observed that not all of the properties of the rate-privacy function carry over from the discrete to the continuous case.
Bearing fault diagnosis based on spectrum images of vibration signals
Venue: arXiv cs.CV 2016
Year: 2016

Bearing fault diagnosis has been a challenge in the monitoring activities of rotating machinery, and it's receiving more and more attention. The conventional fault diagnosis methods usually extract features from the waveforms or spectrums of vibration signals in order to realize fault classification. In this paper, a novel feature in the form of images is presented, namely the spectrum images of vibration signals. The spectrum images are simply obtained by doing fast Fourier transformation. Such images are processed with two-dimensional principal component analysis (2DPCA) to reduce the dimensions, and then a minimum distance method is applied to classify the faults of bearings. The effectiveness of the proposed method is verified with experimental data.
An efficient and secure RSA--like cryptosystem exploiting Rédei rational functions over conics
Venue: arXiv cs.IT 2016
Year: 2016

We define an isomorphism between the group of points of a conic and the set of integers modulo a prime equipped with a non-standard product. This product can be efficiently evaluated through the use of Rédei rational functions. We then exploit the isomorphism to construct a novel RSA-like scheme. We compare our scheme with classic RSA and with RSA-like schemes based on the cubic or conic equation. The decryption operation of the proposed scheme turns to be two times faster than RSA, and involves the lowest number of modular inversions with respect to other RSA-like schemes based on curves. Our solution offers the same security as RSA in a one-to-one communication and more security in broadcast applications.
The Fourier Transform of Poisson Multinomial Distributions and its Algorithmic Applications
Venue: arXiv cs.DS 2016
Year: 2016

An $(n, k)$-Poisson Multinomial Distribution (PMD) is a random variable of the form $X = \sum_{i=1}^n X_i$, where the $X_i$'s are independent random vectors supported on the set of standard basis vectors in $\mathbb{R}^k.$ In this paper, we obtain a refined structural understanding of PMDs by analyzing their Fourier transform. As our core structural result, we prove that the Fourier transform of PMDs is {\em approximately sparse}, i.e., roughly speaking, its $L_1$-norm is small outside a small set. By building on this result, we obtain the following applications: {\bf Learning Theory.} We design the first computationally efficient learning algorithm for PMDs with respect to the total variation distance. Our algorithm learns an arbitrary $(n, k)$-PMD within variation distance $ε$ using a near-optimal sample size of $\widetilde{O}_k(1/ε^2),$ and runs in time $\widetilde{O}_k(1/ε^2) \cdot \log n.$ Previously, no algorithm with a $\mathrm{poly}(1/ε)$ runtime was known, even for $k=3.$ {\bf Game Theory.} We give the first efficient polynomial-time approximation scheme (EPTAS) for computing Nash equilibria in anonymous games. For normalized anonymous games with $n$ players and $k$ strategies, our algorithm computes a well-supported $ε$-Nash equilibrium in time $n^{O(k^3)} \cdot (k/ε)^{O(k^3\log(k/ε)/\log\log(k/ε))^{k-1}}.$ The best previous algorithm for this problem had running time $n^{(f(k)/ε)^k},$ where $f(k) = Ω(k^{k^2})$, for any $k>2.$ {\bf Statistics.} We prove a multivariate central limit theorem (CLT) that relates an arbitrary PMD to a discretized multivariate Gaussian with the same mean and covariance, in total variation distance. Our new CLT strengthens the CLT of Valiant and Valiant by completely removing the dependence on $n$ in the error bound.
A Size-Free CLT for Poisson Multinomials and its Applications
Venue: arXiv cs.DS 2016
Year: 2016

An $(n,k)$-Poisson Multinomial Distribution (PMD) is the distribution of the sum of $n$ independent random vectors supported on the set ${\cal B}_k=\{e_1,\ldots,e_k\}$ of standard basis vectors in $\mathbb{R}^k$. We show that any $(n,k)$-PMD is ${\rm poly}\left({k\over σ}\right)$-close in total variation distance to the (appropriately discretized) multi-dimensional Gaussian with the same first two moments, removing the dependence on $n$ from the Central Limit Theorem of Valiant and Valiant. Interestingly, our CLT is obtained by bootstrapping the Valiant-Valiant CLT itself through the structural characterization of PMDs shown in recent work by Daskalakis, Kamath, and Tzamos. In turn, our stronger CLT can be leveraged to obtain an efficient PTAS for approximate Nash equilibria in anonymous games, significantly improving the state of the art, and matching qualitatively the running time dependence on $n$ and $1/\varepsilon$ of the best known algorithm for two-strategy anonymous games. Our new CLT also enables the construction of covers for the set of $(n,k)$-PMDs, which are proper and whose size is shown to be essentially optimal. Our cover construction combines our CLT with the Shapley-Folkman theorem and recent sparsification results for Laplacian matrices by Batson, Spielman, and Srivastava. Our cover size lower bound is based on an algebraic geometric construction. Finally, leveraging the structural properties of the Fourier spectrum of PMDs we show that these distributions can be learned from $O_k(1/\varepsilon^2)$ samples in ${\rm poly}_k(1/\varepsilon)$-time, removing the quasi-polynomial dependence of the running time on $1/\varepsilon$ from the algorithm of Daskalakis, Kamath, and Tzamos.
Improving performance of recurrent neural network with relu nonlinearity
Venue: arXiv cs.NE 2016
Year: 2016

In recent years significant progress has been made in successfully training recurrent neural networks (RNNs) on sequence learning problems involving long range temporal dependencies. The progress has been made on three fronts: (a) Algorithmic improvements involving sophisticated optimization techniques, (b) network design involving complex hidden layer nodes and specialized recurrent layer connections and (c) weight initialization methods. In this paper, we focus on recently proposed weight initialization with identity matrix for the recurrent weights in a RNN. This initialization is specifically proposed for hidden nodes with Rectified Linear Unit (ReLU) non linearity. We offer a simple dynamical systems perspective on weight initialization process, which allows us to propose a modified weight initialization strategy. We show that this initialization technique leads to successfully training RNNs composed of ReLUs. We demonstrate that our proposal produces comparable or better solution for three toy problems involving long range temporal structure: the addition problem, the multiplication problem and the MNIST classification problem using sequence of pixels. In addition, we present results for a benchmark action recognition problem.
On the Optimal Sample Complexity for Best Arm Identification
Venue: arXiv cs.LG 2016
Year: 2016

We study the best arm identification (BEST-1-ARM) problem, which is defined as follows. We are given $n$ stochastic bandit arms. The $i$th arm has a reward distribution $D_i$ with an unknown mean $μ_{i}$. Upon each play of the $i$th arm, we can get a reward, sampled i.i.d. from $D_i$. We would like to identify the arm with the largest mean with probability at least $1-δ$, using as few samples as possible. We provide a nontrivial algorithm for BEST-1-ARM, which improves upon several prior upper bounds on the same problem. We also study an important special case where there are only two arms, which we call the sign problem. We provide a new lower bound of sign, simplifying and significantly extending a classical result by Farrell in 1964, with a completely new proof. Using the new lower bound for sign, we obtain the first lower bound for BEST-1-ARM that goes beyond the classic Mannor-Tsitsiklis lower bound, by an interesting reduction from Sign to BEST-1-ARM. We propose an interesting conjecture concerning the optimal sample complexity of BEST-1-ARM from the perspective of instance-wise optimality.
Hand-Object Interaction and Precise Localization in Transitive Action Recognition
Venue: arXiv cs.CV 2016
Year: 2016

Action recognition in still images has seen major improvement in recent years due to advances in human pose estimation, object recognition and stronger feature representations produced by deep neural networks. However, there are still many cases in which performance remains far from that of humans. A major difficulty arises in distinguishing between transitive actions in which the overall actor pose is similar, and recognition therefore depends on details of the grasp and the object, which may be largely occluded. In this paper we demonstrate how recognition is improved by obtaining precise localization of the action-object and consequently extracting details of the object shape together with the actor-object interaction. To obtain exact localization of the action object and its interaction with the actor, we employ a coarse-to-fine approach which combines semantic segmentation and contextual features, in successive stages. We focus on (but are not limited) to face-related actions, a set of actions that includes several currently challenging categories. We present an average relative improvement of 35% over state-of-the art and validate through experimentation the effectiveness of our approach.
Novel Feature Extraction, Selection and Fusion for Effective Malware Family Classification
Venue: arXiv cs.CR 2016
Year: 2016

Modern malware is designed with mutation characteristics, namely polymorphism and metamorphism, which causes an enormous growth in the number of variants of malware samples. Categorization of malware samples on the basis of their behaviors is essential for the computer security community, because they receive huge number of malware everyday, and the signature extraction process is usually based on malicious parts characterizing malware families. Microsoft released a malware classification challenge in 2015 with a huge dataset of near 0.5 terabytes of data, containing more than 20K malware samples. The analysis of this dataset inspired the development of a novel paradigm that is effective in categorizing malware variants into their actual family groups. This paradigm is presented and discussed in the present paper, where emphasis has been given to the phases related to the extraction, and selection of a set of novel features for the effective representation of malware samples. Features can be grouped according to different characteristics of malware behavior, and their fusion is performed according to a per-class weighting paradigm. The proposed method achieved a very high accuracy ($\approx$ 0.998) on the Microsoft Malware Challenge dataset.
Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks
Venue: arXiv cs.CR 2016
Year: 2016

Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 10^30. We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.
A Trust Domains Taxonomy for Securely Sharing Information: A Preliminary Investigation
Venue: arXiv cs.CR 2016
Year: 2016

Information sharing has become a vital part in our day-to-day life due to the pervasiveness of Internet technology. In any given collaboration, information needs to flow from one participant to another. While participants may be interested in sharing information with one another, it is often necessary for them to establish the impact of sharing certain kinds of information. This is because certain information could have detrimental effects when it ends up in wrong hands. For this reason, any would-be participant in a given collaboration may need to establish the guarantees that the collaboration provides, in terms of protecting sensitive information, before joining the collaboration as well as evaluating the impact of sharing a given piece of information with a given set of entities. In order to address this issue, earlier work introduced a trust domains taxonomy that aims at managing trust-related issues in information sharing. This paper attempts to empirically investigate the proposed taxonomy through a possible scenario (e.g. the ConfiChair system). The study results determined that Role, Policy, Action, Control, Evidence and Asset elements should be incorporated into the taxonomy for securely sharing information among others. Additionally, the study results showed that the ConfiChair, a novel cloud-based conference management system, offers strong privacy and confidentiality guarantees.
Word Embedding based Correlation Model for Question/Answer Matching
Venue: arXiv cs.CL 2016
Year: 2016

With the development of community based question answering (Q&A) services, a large scale of Q&A archives have been accumulated and are an important information and knowledge resource on the web. Question and answer matching has been attached much importance to for its ability to reuse knowledge stored in these systems: it can be useful in enhancing user experience with recurrent questions. In this paper, we try to improve the matching accuracy by overcoming the lexical gap between question and answer pairs. A Word Embedding based Correlation (WEC) model is proposed by integrating advantages of both the translation model and word embedding, given a random pair of words, WEC can score their co-occurrence probability in Q&A pairs and it can also leverage the continuity and smoothness of continuous space word representation to deal with new pairs of words that are rare in the training parallel text. An experimental study on Yahoo! Answers dataset and Baidu Zhidao dataset shows this new method's promising potential.
Sensor Scheduling in Variance Based Event Triggered Estimation with Packet Drops
Venue: arXiv cs.SY 2016
Year: 2016

This paper considers a remote state estimation problem with multiple sensors observing a dynamical process, where sensors transmit local state estimates over an independent and identically distributed (i.i.d.) packet dropping channel to a remote estimator. At every discrete time instant, the remote estimator decides whether each sensor should transmit or not, with each sensor transmission incurring a fixed energy cost. The channel is shared such that collisions will occur if more than one sensor transmits at a time. Performance is quantified via an optimization problem that minimizes a convex combination of the expected estimation error covariance at the remote estimator and expected energy usage across the sensors. For transmission schedules dependent only on the estimation error covariance at the remote estimator, this work establishes structural results on the optimal scheduling which show that 1) for unstable systems, if the error covariance is large then a sensor will always be scheduled to transmit, and 2) there is a threshold-type behaviour in switching from one sensor transmitting to another. Specializing to the single sensor case, these structural results demonstrate that a threshold policy (i.e. transmit if the error covariance exceeds a certain threshold and don't transmit otherwise) is optimal. We also consider the situation where sensors transmit measurements instead of state estimates, and establish structural results including the optimality of threshold policies for the single sensor, scalar case. These results provide a theoretical justification for the use of such threshold policies in variance based event triggered estimation. Numerical studies confirm the qualitative behaviour predicted by our structural results. An extension of the structural results to Markovian packet drops is also outlined.
A discontinuous Galerkin Method for the EEG Forward Problem using the Subtraction Approach
Venue: arXiv cs.CE 2016
Year: 2016

In order to perform electroencephalography (EEG) source reconstruction, i.e., to localize the sources underlying a measured EEG, the electric potential distribution at the electrodes generated by a dipolar current source in the brain has to be simulated, which is the so-called EEG forward problem. To solve it accurately, it is necessary to apply numerical methods that are able to take the individual geometry and conductivity distribution of the subject's head into account. In this context, the finite element method (FEM) has shown high numerical accuracy with the possibility to model complex geometries and conductive features, e.g., white matter conductivity anisotropy. In this article, we introduce and analyze the application of a discontinuous Galerkin (DG) method, a finite element method that includes features of the finite volume framework, to the EEG forward problem. The DG-FEM approach fulfills the conservation property of electric charge also in the discrete case, making it attractive for a variety of applications. Furthermore, as we show, this approach can alleviate modeling inaccuracies that might occur in head geometries when using classical FE methods, e.g., so-called "skull leakage effects", which may occur in areas where the thickness of the skull is in the range of the mesh resolution. Therefore, we derive a DG formulation of the FEM subtraction approach for the EEG forward problem and present numerical results that highlight the advantageous features and the potential benefits of the proposed approach.
Efficient AUC Optimization for Information Ranking Applications
Venue: arXiv cs.IR 2016
Year: 2016

Adequate evaluation of an information retrieval system to estimate future performance is a crucial task. Area under the ROC curve (AUC) is widely used to evaluate the generalization of a retrieval system. However, the objective function optimized in many retrieval systems is the error rate and not the AUC value. This paper provides an efficient and effective non-linear approach to optimize AUC using additive regression trees, with a special emphasis on the use of multi-class AUC (MAUC) because multiple relevance levels are widely used in many ranking applications. Compared to a conventional linear approach, the performance of the non-linear approach is comparable on binary-relevance benchmark datasets and is better on multi-relevance benchmark datasets.
Counting Ones Without Broadword Operations
Venue: arXiv cs.CC 2016
Year: 2016

A lower time bound $Ω(\min(ν(x), n-ν(x))$ for counting the number of ones in a binary input word $x$ of length $n$ is presented, where $ν(x)$ is the number of ones. The operations available are increment, decrement, bit-wise logical operations, and assignment. The only constant available is zero. An almost matching upper bound is also obtained.
Recurrent Neural Networks Hardware Implementation on FPGA
Venue: arXiv cs.NE 2016
Year: 2016

Recurrent Neural Networks (RNNs) have the ability to retain memory and learn data sequences. Due to the recurrent nature of RNNs, it is sometimes hard to parallelize all its computations on conventional hardware. CPUs do not currently offer large parallelism, while GPUs offer limited parallelism due to sequential components of RNN models. In this paper we present a hardware implementation of Long-Short Term Memory (LSTM) recurrent network on the programmable logic Zynq 7020 FPGA from Xilinx. We implemented a RNN with $2$ layers and $128$ hidden units in hardware and it has been tested using a character level language model. The implementation is more than $21\times$ faster than the ARM CPU embedded on the Zynq 7020 FPGA. This work can potentially evolve to a RNN co-processor for future mobile devices.