OpenRaiser/Intern-Atlas huggingface dataset:
Sparse Neural Networks Topologies
Venue: arXiv cs.LG 2017
Year: 2017

We propose Sparse Neural Network architectures that are based on random or structured bipartite graph topologies. Sparse architectures provide compression of the models learned and speed-ups of computations, they can also surpass their unstructured or fully connected counterparts. As we show, even more compact topologies of the so-called SNN (Sparse Neural Network) can be achieved with the use of structured graphs of connections between consecutive layers of neurons. In this paper, we investigate how the accuracy and training speed of the models depend on the topology and sparsity of the neural network. Previous approaches using sparcity are all based on fully connected neural network models and create sparcity during training phase, instead we explicitly define a sparse architectures of connections before the training. Building compact neural network models is coherent with empirical observations showing that there is much redundancy in learned neural network models. We show experimentally that the accuracy of the models learned with neural networks depends on expander-like properties of the underlying topologies such as the spectral gap and algebraic connectivity rather than the density of the graphs of connections.
SuperMinHash - A New Minwise Hashing Algorithm for Jaccard Similarity Estimation
Venue: arXiv cs.DS 2017
Year: 2017

This paper presents a new algorithm for calculating hash signatures of sets which can be directly used for Jaccard similarity estimation. The new approach is an improvement over the MinHash algorithm, because it has a better runtime behavior and the resulting signatures allow a more precise estimation of the Jaccard index.
Addressing Item-Cold Start Problem in Recommendation Systems using Model Based Approach and Deep Learning
Venue: arXiv cs.IR 2017
Year: 2017

Traditional recommendation systems rely on past usage data in order to generate new recommendations. Those approaches fail to generate sensible recommendations for new users and items into the system due to missing information about their past interactions. In this paper, we propose a solution for successfully addressing item-cold start problem which uses model-based approach and recent advances in deep learning. In particular, we use latent factor model for recommendation, and predict the latent factors from item's descriptions using convolutional neural network when they cannot be obtained from usage data. Latent factors obtained by applying matrix factorization to the available usage data are used as ground truth to train the convolutional neural network. To create latent factor representations for the new items, the convolutional neural network uses their textual description. The results from the experiments reveal that the proposed approach significantly outperforms several baseline estimators.
Look No Further: Adapting the Localization Sensory Window to the Temporal Characteristics of the Environment
Venue: arXiv cs.RO 2017
Year: 2017

Many localization algorithms use a spatiotemporal window of sensory information in order to recognize spatial locations, and the length of this window is often a sensitive parameter that must be tuned to the specifics of the application. This letter presents a general method for environment-driven variation of the length of the spatiotemporal window based on searching for the most significant localization hypothesis, to use as much context as is appropriate but not more. We evaluate this approach on benchmark datasets using visual and Wi-Fi sensor modalities and a variety of sensory comparison front-ends under in-order and out-of-order traversals of the environment. Our results show that the system greatly reduces the maximum distance traveled without localization compared to a fixed-length approach while achieving competitive localization accuracy, and our proposed method achieves this performance without deployment-time tuning.
Kapre: On-GPU Audio Preprocessing Layers for a Quick Implementation of Deep Neural Network Models with Keras
Venue: arXiv cs.SD 2017
Year: 2017

We introduce Kapre, Keras layers for audio and music signal preprocessing. Music research using deep neural networks requires a heavy and tedious preprocessing stage, for which audio processing parameters are often ignored in parameter optimisation. To solve this problem, Kapre implements time-frequency conversions, normalisation, and data augmentation as Keras layers. We report simple benchmark results, showing real-time on-GPU preprocessing adds a reasonable amount of computation.
Channels with Cooperation Links that May Be Absent
Venue: arXiv cs.IT 2017
Year: 2017

It is well known that cooperation between users in a communication network can lead to significant performance gains. A common assumption in past works is that all the users are aware of the resources available for cooperation, and know exactly to what extent these resources can be used. Unfortunately, in many modern communication networks the availability of cooperation links cannot be guaranteed a priori, due to the dynamic nature of the network. In this work a family of models is suggested where the cooperation links may or may not be present. Coding schemes are devised that exploit the cooperation links if they are present, and can still operate (although at reduced rates) if cooperation is not possible.
Computing the channel capacity of a communication system affected by uncertain transition probabilities
Venue: arXiv cs.IT 2017
Year: 2017

We study the problem of computing the capacity of a discrete memoryless channel under uncertainty affecting the channel law matrix, and possibly with a constraint on the average cost of the input distribution. The problem has been formulated in the literature as a max-min problem. We use the robust optimization methodology to convert the max-min problem to a standard convex optimization problem. For small-sized problems, and for many types of uncertainty, such a problem can be solved in principle using interior point methods (IPM). However, for large-scale problems, IPM are not practical. Here, we suggest an $\mathcal{O}(1/T)$ first-order algorithm based on Nemirovski (2004) which is applied directly to the max-min problem.
Combinatorial Properties and Recognition of Unit Square Visibility Graphs
Venue: arXiv cs.CG 2017
Year: 2017

Unit square (grid) visibility graphs (USV and USGV, resp.) are described by axis-parallel visibility between unit squares placed (on integer grid coordinates) in the plane. We investigate combinatorial properties of these graph classes and the hardness of variants of the recognition problem, i.e., the problem of representing USGV with fixed visibilities within small area and, for USV, the general recognition problem.
"Everything I Disagree With is #FakeNews": Correlating Political Polarization and Spread of Misinformation
Venue: arXiv cs.SI 2017
Year: 2017

An important challenge in the process of tracking and detecting the dissemination of misinformation is to understand the political gap between people that engage with the so called "fake news". A possible factor responsible for this gap is opinion polarization, which may prompt the general public to classify content that they disagree or want to discredit as fake. In this work, we study the relationship between political polarization and content reported by Twitter users as related to "fake news". We investigate how polarization may create distinct narratives on what misinformation actually is. We perform our study based on two datasets collected from Twitter. The first dataset contains tweets about US politics in general, from which we compute the degree of polarization of each user towards the Republican and Democratic Party. In the second dataset, we collect tweets and URLs that co-occurred with "fake news" related keywords and hashtags, such as #FakeNews and #AlternativeFact, as well as reactions towards such tweets and URLs. We then analyze the relationship between polarization and what is perceived as misinformation, and whether users are designating information that they disagree as fake. Our results show an increase in the polarization of users and URLs associated with fake-news keywords and hashtags, when compared to information not labeled as "fake news". We discuss the impact of our findings on the challenges of tracking "fake news" in the ongoing battle against misinformation.
Bayesian Joint Modelling for Object Localisation in Weakly Labelled Images
Venue: arXiv cs.CV 2017
Year: 2017

We address the problem of localisation of objects as bounding boxes in images and videos with weak labels. This weakly supervised object localisation problem has been tackled in the past using discriminative models where each object class is localised independently from other classes. In this paper, a novel framework based on Bayesian joint topic modelling is proposed, which differs significantly from the existing ones in that: (1) All foreground object classes are modelled jointly in a single generative model that encodes multiple object co-existence so that "explaining away" inference can resolve ambiguity and lead to better learning and localisation. (2) Image backgrounds are shared across classes to better learn varying surroundings and "push out" objects of interest. (3) Our model can be learned with a mixture of weakly labelled and unlabelled data, allowing the large volume of unlabelled images on the Internet to be exploited for learning. Moreover, the Bayesian formulation enables the exploitation of various types of prior knowledge to compensate for the limited supervision offered by weakly labelled data, as well as Bayesian domain adaptation for transfer learning. Extensive experiments on the PASCAL VOC, ImageNet and YouTube-Object videos datasets demonstrate the effectiveness of our Bayesian joint model for weakly supervised object localisation.
Event Analysis of Pulse-reclosers in Distribution Systems Through Sparse Representation
Venue: arXiv cs.SY 2017
Year: 2017

The pulse-recloser uses pulse testing technology to verify that the line is clear of faults before initiating a reclose operation, which significantly reduces stress on the system components (e.g. substation transformers) and voltage sags on adjacent feeders. Online event analysis of pulse-reclosers are essential to increases the overall utility of the devices, especially when there are numerous devices installed throughout the distribution system. In this paper, field data recorded from several devices were analyzed to identify specific activity and fault locations. An algorithm is developed to screen the data to identify the status of each pole and to tag time windows with a possible pulse event. In the next step, selected time windows are further analyzed and classified using a sparse representation technique by solving an l1-regularized least-square problem. This classification is obtained by comparing the pulse signature with the reference dictionary to find a set that most closely matches the pulse features. This work also sheds additional light on the possibility of fault classification based on the pulse signature. Field data collected from a distribution system are used to verify the effectiveness and reliability of the proposed method.
Scalable Co-Optimization of Morphology and Control in Embodied Machines
Venue: arXiv cs.AI 2017
Year: 2017

Evolution sculpts both the body plans and nervous systems of agents together over time. In contrast, in AI and robotics, a robot's body plan is usually designed by hand, and control policies are then optimized for that fixed design. The task of simultaneously co-optimizing the morphology and controller of an embodied robot has remained a challenge. In psychology, the theory of embodied cognition posits that behavior arises from a close coupling between body plan and sensorimotor control, which suggests why co-optimizing these two subsystems is so difficult: most evolutionary changes to morphology tend to adversely impact sensorimotor control, leading to an overall decrease in behavioral performance. Here, we further examine this hypothesis and demonstrate a technique for "morphological innovation protection", which temporarily reduces selection pressure on recently morphologically-changed individuals, thus enabling evolution some time to "readapt" to the new morphology with subsequent control policy mutations. We show the potential for this method to avoid local optima and converge to similar highly fit morphologies across widely varying initial conditions, while sustaining fitness improvements further into optimization. While this technique is admittedly only the first of many steps that must be taken to achieve scalable optimization of embodied machines, we hope that theoretical insight into the cause of evolutionary stagnation in current methods will help to enable the automation of robot design and behavioral training -- while simultaneously providing a testbed to investigate the theory of embodied cognition.
ESCAPE - Echo SCraper and ClAssifier of PErsons: A novel tool to facilitate using voice-controlled devices for research
Venue: arXiv cs.HC 2017
Year: 2017

Smart devices have become common place in many homes, and these devices can be utilized to provide support for people with mental or physical deficits. Voice-controlled assistants are a class of smart device that collect a large amount of data in the home. In this work we present Echo SCraper and ClAssifier of Persons (ESCAPE), an open source software for the extraction of Amazon Echo interaction data, and speaker recognition on that data. We show that ESCAPE is able to extract data from a voice-controlled assistant and classify with accuracy who is talking, based on a small number of labeled audio data. Using ESCAPE to extract interactions recorded over 3 months in the first author's home yields a rich dataset of transcribed audio recordings. Our results demonstrate that using this software the Amazon Echo can be used to study participants in a naturalistic setting with minimal intrusion. We also discuss the potential for usage of voice-controlled devices together with ESCAPE to understand how diseases affect individuals, and how these data can be used to monitor disease processes in general.
On the combinatorics of the 2-class classification problem
Venue: arXiv cs.DM 2017
Year: 2017

A set of points $X = X_B \cup X_R \subseteq \mathbb{R}^d$ is linearly separable if the convex hulls of $X_B$ and $X_R$ are disjoint, hence there exists a hyperplane separating $X_B$ from $X_R$. Such a hyperplane provides a method for classifying new points, according to which side of the hyperplane the new points lie. When such a linear separation is not possible, it may still be possible to partition $X_B$ and $X_R$ into prespecified numbers of groups, in such a way that every group from $X_B$ is linearly separable from every group from $X_R$. We may also discard some points as outliers, and seek to minimize the number of outliers necessary to find such a partition. Based on these ideas, Bertsimas and Shioda proposed the classification and regression by integer optimization (CRIO) method in 2007. In this work we explore the integer programming aspects of the classification part of CRIO, in particular theoretical properties of the associated formulation. We are able to find facet-inducing inequalities coming from the stable set polytope, hence showing that this classification problem has exploitable combinatorial properties.
Hybrid Spatio-Temporal Artificial Noise Design for Secure MIMOME-OFDM Systems
Venue: arXiv cs.IT 2017
Year: 2017

This paper investigates artificial noise injection into the temporal and spatial dimensions of a legitimate wireless communication system to secure its transmissions from potential eavesdropping. We consider a multiple-input single-output (MISO) orthogonal frequency division multiplexing (OFDM) system in the presence of a single-antenna passive eavesdropper and derive both the secrecy rate and average secrecy rate of the legitimate system. It is assumed that the legitimate transmitter knows the full channel information of the legitimate transceivers but does not know the instantaneous channel state information of the passive eavesdropper. Closed-form expressions for the secrecy rate and average secrecy rate are derived for the asymptotic case with a large number of transmit antennas. We also investigate 1) the power allocation between the data and the AN; 2) the power allocation between the spatial and the temporal AN. Computer simulations are carried out to evaluate the performance of our proposed artificial noise scheme.
Low Resolution Face Recognition Using a Two-Branch Deep Convolutional Neural Network Architecture
Venue: arXiv cs.CV 2017
Year: 2017

We propose a novel couple mappings method for low resolution face recognition using deep convolutional neural networks (DCNNs). The proposed architecture consists of two branches of DCNNs to map the high and low resolution face images into a common space with nonlinear transformations. The branch corresponding to transformation of high resolution images consists of 14 layers and the other branch which maps the low resolution face images to the common space includes a 5-layer super-resolution network connected to a 14-layer network. The distance between the features of corresponding high and low resolution images are backpropagated to train the networks. Our proposed method is evaluated on FERET data set and compared with state-of-the-art competing methods. Our extensive experimental results show that the proposed method significantly improves the recognition performance especially for very low resolution probe face images (11.4% improvement in recognition accuracy). Furthermore, it can reconstruct a high resolution image from its corresponding low resolution probe image which is comparable with state-of-the-art super-resolution methods in terms of visual quality.
KeyDrown: Eliminating Keystroke Timing Side-Channel Attacks
Venue: arXiv cs.CR 2017
Year: 2017

Besides cryptographic secrets, side-channel attacks also leak sensitive user input. The most accurate attacks exploit cache timings or interrupt information to monitor keystroke timings and subsequently infer typed words and sentences. Previously proposed countermeasures fail to prevent keystroke timing attacks as they do not protect keystroke processing among the entire software stack. We close this gap with KeyDrown, a new defense mechanism against keystroke timing attacks. KeyDrown injects a large number of fake keystrokes in the kernel to prevent interrupt-based attacks and Prime+Probe attacks on the kernel. All keystrokes, including fake keystrokes, are carefully propagated through the shared library in order to hide any cache activity and thus to prevent Flush+Reload attacks. Finally, we provide additional protection against Prime+Probe for password input in user space programs. We show that attackers cannot distinguish fake keystrokes from real keystrokes anymore and we evaluate KeyDrown on a commodity notebook as well as on two Android smartphones. We show that KeyDrown eliminates any advantage an attacker can gain from using interrupt or cache side-channel information.
Distributed PCP Theorems for Hardness of Approximation in P
Venue: arXiv cs.CC 2017
Year: 2017

We present a new distributed model of probabilistically checkable proofs (PCP). A satisfying assignment $x \in \{0,1\}^n$ to a CNF formula $φ$ is shared between two parties, where Alice knows $x_1, \dots, x_{n/2}$, Bob knows $x_{n/2+1},\dots,x_n$, and both parties know $φ$. The goal is to have Alice and Bob jointly write a PCP that $x$ satisfies $φ$, while exchanging little or no information. Unfortunately, this model as-is does not allow for nontrivial query complexity. Instead, we focus on a non-deterministic variant, where the players are helped by Merlin, a third party who knows all of $x$. Using our framework, we obtain, for the first time, PCP-like reductions from the Strong Exponential Time Hypothesis (SETH) to approximation problems in P. In particular, under SETH we show that there are no truly-subquadratic approximation algorithms for Bichromatic Maximum Inner Product over {0,1}-vectors, Bichromatic LCS Closest Pair over permutations, Approximate Regular Expression Matching, and Diameter in Product Metric. All our inapproximability factors are nearly-tight. In particular, for the first two problems we obtain nearly-polynomial factors of $2^{(\log n)^{1-o(1)}}$; only $(1+o(1))$-factor lower bounds (under SETH) were known before.
The Compressed Model of Residual CNDS
Venue: arXiv cs.CV 2017
Year: 2017

Convolutional neural networks have achieved a great success in the recent years. Although, the way to maximize the performance of the convolutional neural networks still in the beginning. Furthermore, the optimization of the size and the time that need to train the convolutional neural networks is very far away from reaching the researcher's ambition. In this paper, we proposed a new convolutional neural network that combined several techniques to boost the optimization of the convolutional neural network in the aspects of speed and size. As we used our previous model Residual-CNDS (ResCNDS), which solved the problems of slower convergence, overfitting, and degradation, and compressed it. The outcome model called Residual-Squeeze-CNDS (ResSquCNDS), which we demonstrated on our sold technique to add residual learning and our model of compressing the convolutional neural networks. Our model of compressing adapted from the SQUEEZENET model, but our model is more generalizable, which can be applied almost to any neural network model, and fully integrated into the residual learning, which addresses the problem of the degradation very successfully. Our proposed model trained on very large-scale MIT Places365-Standard scene datasets, which backing our hypothesis that the new compressed model inherited the best of the previous ResCNDS8 model, and almost get the same accuracy in the validation Top-1 and Top-5 with 87.64% smaller in size and 13.33% faster in the training time.
An online sequence-to-sequence model for noisy speech recognition
Venue: arXiv cs.CL 2017
Year: 2017

Generative models have long been the dominant approach for speech recognition. The success of these models however relies on the use of sophisticated recipes and complicated machinery that is not easily accessible to non-practitioners. Recent innovations in Deep Learning have given rise to an alternative - discriminative models called Sequence-to-Sequence models, that can almost match the accuracy of state of the art generative models. While these models are easy to train as they can be trained end-to-end in a single step, they have a practical limitation that they can only be used for offline recognition. This is because the models require that the entirety of the input sequence be available at the beginning of inference, an assumption that is not valid for instantaneous speech recognition. To address this problem, online sequence-to-sequence models were recently introduced. These models are able to start producing outputs as data arrives, and the model feels confident enough to output partial transcripts. These models, like sequence-to-sequence are causal - the output produced by the model until any time, $t$, affects the features that are computed subsequently. This makes the model inherently more powerful than generative models that are unable to change features that are computed from the data. This paper highlights two main contributions - an improvement to online sequence-to-sequence model training, and its application to noisy settings with mixed speech from two speakers.