OpenRaiser/Intern-Atlas huggingface dataset:
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Venue: arXiv cs.CL 2017
Year: 2017

Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to learn a different vector for each sense of a word, by exploiting crosslingual signals to aid sense identification. We present a multi-view Bayesian non-parametric algorithm which improves multi-sense word embeddings by (a) using multilingual (i.e., more than two languages) corpora to significantly improve sense embeddings beyond what one achieves with bilingual information, and (b) uses a principled approach to learn a variable number of senses per word, in a data-driven manner. Ours is the first approach with the ability to leverage multilingual corpora efficiently for multi-sense representation learning. Experiments show that multilingual training significantly improves performance over monolingual and bilingual training, by allowing us to combine different parallel corpora to leverage multilingual context. Multilingual training yields comparable performance to a state of the art mono-lingual model trained on five times more training data.
Automated text summarisation and evidence-based medicine: A survey of two domains
Venue: arXiv cs.CL 2017
Year: 2017

The practice of evidence-based medicine (EBM) urges medical practitioners to utilise the latest research evidence when making clinical decisions. Because of the massive and growing volume of published research on various medical topics, practitioners often find themselves overloaded with information. As such, natural language processing research has recently commenced exploring techniques for performing medical domain-specific automated text summarisation (ATS) techniques-- targeted towards the task of condensing large medical texts. However, the development of effective summarisation techniques for this task requires cross-domain knowledge. We present a survey of EBM, the domain-specific needs for EBM, automated summarisation techniques, and how they have been applied hitherto. We envision that this survey will serve as a first resource for the development of future operational text summarisation techniques for EBM.
Detecting Integrity Attacks on Control Systems using a Moving Target Approach
Venue: arXiv cs.SY 2017
Year: 2017

Maintaining the security of control systems in the presence of integrity attacks is a significant challenge. In literature, several possible attacks against control systems have been formulated including replay, false data injection, and zero dynamics attacks. The detection and prevention of these attacks may require the defender to possess a particular subset of trusted communication channels. Alternatively, these attacks can be prevented by keeping the system model secret from the adversary. In this paper, we consider an adversary who has the ability to modify and read all sensor and actuator channels. To thwart this adversary, we introduce external states dependent on the state of the control system, with linear time-varying dynamics unknown to the adversary. We also include sensors to measure these states. The presence of unknown time-varying dynamics is leveraged to detect an adversary who simultaneously aims to identify the system and inject stealthy outputs. Potential attack strategies and bounds on the attacker's performance are provided.
End-to-end Learning of Image based Lane-Change Decision
Venue: arXiv cs.CV 2017
Year: 2017

We propose an image based end-to-end learning framework that helps lane-change decisions for human drivers and autonomous vehicles. The proposed system, Safe Lane-Change Aid Network (SLCAN), trains a deep convolutional neural network to classify the status of adjacent lanes from rear view images acquired by cameras mounted on both sides of the vehicle. Rather than depending on any explicit object detection or tracking scheme, SLCAN reads the whole input image and directly decides whether initiation of the lane-change at the moment is safe or not. We collected and annotated 77,273 rear side view images to train and test SLCAN. Experimental results show that the proposed framework achieves 96.98% classification accuracy although the test images are from unseen roadways. We also visualize the saliency map to understand which part of image SLCAN looks at for correct decisions.
Do GANs actually learn the distribution? An empirical study
Venue: arXiv cs.LG 2017
Year: 2017

Do GANS (Generative Adversarial Nets) actually learn the target distribution? The foundational paper of (Goodfellow et al 2014) suggested they do, if they were given sufficiently large deep nets, sample size, and computation time. A recent theoretical analysis in Arora et al (to appear at ICML 2017) raised doubts whether the same holds when discriminator has finite size. It showed that the training objective can approach its optimum value even if the generated distribution has very low support ---in other words, the training objective is unable to prevent mode collapse. The current note reports experiments suggesting that such problems are not merely theoretical. It presents empirical evidence that well-known GANs approaches do learn distributions of fairly low support, and thus presumably are not learning the target distribution. The main technical contribution is a new proposed test, based upon the famous birthday paradox, for estimating the support size of the generated distribution.
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Venue: arXiv cs.CV 2017
Year: 2017

Skeleton-based human action recognition has attracted a lot of research attention during the past few years. Recent works attempted to utilize recurrent neural networks to model the temporal dependencies between the 3D positional configurations of human body joints for better analysis of human activities in the skeletal data. The proposed work extends this idea to spatial domain as well as temporal domain to better analyze the hidden sources of action-related information within the human skeleton sequences in both of these domains simultaneously. Based on the pictorial structure of Kinect's skeletal data, an effective tree-structure based traversal framework is also proposed. In order to deal with the noise in the skeletal data, a new gating mechanism within LSTM module is introduced, with which the network can learn the reliability of the sequential data and accordingly adjust the effect of the input data on the updating procedure of the long-term context representation stored in the unit's memory cell. Moreover, we introduce a novel multi-modal feature fusion strategy within the LSTM unit in this paper. The comprehensive experimental results on seven challenging benchmark datasets for human action recognition demonstrate the effectiveness of the proposed method.
Bayesian Brain meets Bayesian Recommender - Towards Systems with Empathy for the Human Nature
Venue: arXiv cs.HC 2017
Year: 2017

In this paper we consider the modern theory of the Bayesian brain from cognitive neurosciences in the light of recommender systems and expose potentials for our community. In particular, we elaborate on noisy user feedback and the thus resulting multicomponent user models, which have indeed a biological origin. In real user experiments we observe the impact of both factors directly in a repeated rating task along with recommendation. As a consequence, this contribution supports the plausibility of contemporary theories of mind in the context of recommender systems and can be understood as a solicitation to integrate ideas of cognitive neurosciences into our systems in order to further improve the prediction of human behaviour.
A Meta-Learning Approach to One-Step Active Learning
Venue: arXiv cs.LG 2017
Year: 2017

We consider the problem of learning when obtaining the training labels is costly, which is usually tackled in the literature using active-learning techniques. These approaches provide strategies to choose the examples to label before or during training. These strategies are usually based on heuristics or even theoretical measures, but are not learned as they are directly used during training. We design a model which aims at \textit{learning active-learning strategies} using a meta-learning setting. More specifically, we consider a pool-based setting, where the system observes all the examples of the dataset of a problem and has to choose the subset of examples to label in a single shot. Experiments show encouraging results.
On the Energy Efficiency of Rate and Transmission Power Control in 802.11
Venue: arXiv cs.NI 2017
Year: 2017

Rate adaptation and transmission power control in 802.11 WLANs have received a lot of attention from the research community, with most of the proposals aiming at maximising throughput based on network conditions. Considering energy consumption, an implicit assumption is that optimality in throughput implies optimality in energy efficiency, but this assumption has been recently put into question. In this paper, we address via analysis, simulation and experimentation the relation between throughput performance and energy efficiency in multi-rate 802.11 scenarios. We demonstrate the trade-off between these performance figures, confirming that they may not be simultaneously optimised, and analyse their sensitivity towards the energy consumption parameters of the device. We analyse this trade-off in existing rate adaptation with transmission power control algorithms, and discuss how to design novel schemes taking energy consumption into account.
StreamLearner: Distributed Incremental Machine Learning on Event Streams: Grand Challenge
Venue: arXiv cs.DC 2017
Year: 2017

Today, massive amounts of streaming data from smart devices need to be analyzed automatically to realize the Internet of Things. The Complex Event Processing (CEP) paradigm promises low-latency pattern detection on event streams. However, CEP systems need to be extended with Machine Learning (ML) capabilities such as online training and inference in order to be able to detect fuzzy patterns (e.g., outliers) and to improve pattern recognition accuracy during runtime using incremental model training. In this paper, we propose a distributed CEP system denoted as StreamLearner for ML-enabled complex event detection. The proposed programming model and data-parallel system architecture enable a wide range of real-world applications and allow for dynamically scaling up and out system resources for low-latency, high-throughput event processing. We show that the DEBS Grand Challenge 2017 case study (i.e., anomaly detection in smart factories) integrates seamlessly into the StreamLearner API. Our experiments verify scalability and high event throughput of StreamLearner.
Towards the Evolution of Multi-Layered Neural Networks: A Dynamic Structured Grammatical Evolution Approach
Venue: arXiv cs.NE 2017
Year: 2017

Current grammar-based NeuroEvolution approaches have several shortcomings. On the one hand, they do not allow the generation of Artificial Neural Networks (ANNs composed of more than one hidden-layer. On the other, there is no way to evolve networks with more than one output neuron. To properly evolve ANNs with more than one hidden-layer and multiple output nodes there is the need to know the number of neurons available in previous layers. In this paper we introduce Dynamic Structured Grammatical Evolution (DSGE): a new genotypic representation that overcomes the aforementioned limitations. By enabling the creation of dynamic rules that specify the connection possibilities of each neuron, the methodology enables the evolution of multi-layered ANNs with more than one output neuron. Results in different classification problems show that DSGE evolves effective single and multi-layered ANNs, with a varying number of output neurons.
The Bernays-Schönfinkel-Ramsey Fragment with Bounded Difference Constraints over the Reals is Decidable
Venue: arXiv cs.LO 2017
Year: 2017

First-order linear real arithmetic enriched with uninterpreted predicate symbols yields an interesting modeling language. However, satisfiability of such formulas is undecidable, even if we restrict the uninterpreted predicate symbols to arity one. In order to find decidable fragments of this language, it is necessary to restrict the expressiveness of the arithmetic part. One possible path is to confine arithmetic expressions to difference constraints of the form $x - y \mathrel{\#} c$, where $\#$ ranges over the standard relations $<, \leq, =, \neq, \geq, >$ and $x,y$ are universally quantified. However, it is known that combining difference constraints with uninterpreted predicate symbols yields an undecidable satisfiability problem again. In this paper, it is shown that satisfiability becomes decidable if we in addition bound the ranges of universally quantified variables. As bounded intervals over the reals still comprise infinitely many values, a trivial instantiation procedure is not sufficient to solve the problem.
Learning Local Feature Aggregation Functions with Backpropagation
Venue: arXiv cs.LG 2017
Year: 2017

This paper introduces a family of local feature aggregation functions and a novel method to estimate their parameters, such that they generate optimal representations for classification (or any task that can be expressed as a cost function minimization problem). To achieve that, we compose the local feature aggregation function with the classifier cost function and we backpropagate the gradient of this cost function in order to update the local feature aggregation function parameters. Experiments on synthetic datasets indicate that our method discovers parameters that model the class-relevant information in addition to the local feature space. Further experiments on a variety of motion and visual descriptors, both on image and video datasets, show that our method outperforms other state-of-the-art local feature aggregation functions, such as Bag of Words, Fisher Vectors and VLAD, by a large margin.
Robust Sonar ATR Through Bayesian Pose Corrected Sparse Classification
Venue: arXiv cs.CV 2017
Year: 2017

Sonar imaging has seen vast improvements over the last few decades due in part to advances in synthetic aperture Sonar (SAS). Sophisticated classification techniques can now be used in Sonar automatic target recognition (ATR) to locate mines and other threatening objects. Among the most promising of these methods is sparse reconstruction-based classification (SRC) which has shown an impressive resiliency to noise, blur, and occlusion. We present a coherent strategy for expanding upon SRC for Sonar ATR that retains SRC's robustness while also being able to handle targets with diverse geometric arrangements, bothersome Rayleigh noise, and unavoidable background clutter. Our method, pose corrected sparsity (PCS), incorporates a novel interpretation of a spike and slab probability distribution towards use as a Bayesian prior for class-specific discrimination in combination with a dictionary learning scheme for localized patch extractions. Additionally, PCS offers the potential for anomaly detection in order to avoid false identifications of tested objects from outside the training set with no additional training required. Compelling results are shown using a database provided by the United States Naval Surface Warfare Center.
Domain reduction techniques for global NLP and MINLP optimization
Venue: arXiv cs.DS 2017
Year: 2017

Optimization solvers routinely utilize presolve techniques, including model simplification, reformulation and domain reduction techniques. Domain reduction techniques are especially important in speeding up convergence to the global optimum for challenging nonconvex nonlinear programming (NLP) and mixed-integer nonlinear programming (MINLP) optimization problems. In this work, we survey the various techniques used for domain reduction of NLP and MINLP optimization problems. We also present a computational analysis of the impact of these techniques on the performance of various widely available global solvers on a collection of 1740 test problems.
Memory-augmented Chinese-Uyghur Neural Machine Translation
Venue: arXiv cs.CL 2017
Year: 2017

Neural machine translation (NMT) has achieved notable performance recently. However, this approach has not been widely applied to the translation task between Chinese and Uyghur, partly due to the limited parallel data resource and the large proportion of rare words caused by the agglutinative nature of Uyghur. In this paper, we collect ~200,000 sentence pairs and show that with this middle-scale database, an attention-based NMT can perform very well on Chinese-Uyghur/Uyghur-Chinese translation. To tackle rare words, we propose a novel memory structure to assist the NMT inference. Our experiments demonstrated that the memory-augmented NMT (M-NMT) outperforms both the vanilla NMT and the phrase-based statistical machine translation (SMT). Interestingly, the memory structure provides an elegant way for dealing with words that are out of vocabulary.
Controlled Tactile Exploration and Haptic Object Recognition
Venue: arXiv cs.RO 2017
Year: 2017

In this paper we propose a novel method for in-hand object recognition. The method is composed of a grasp stabilization controller and two exploratory behaviours to capture the shape and the softness of an object. Grasp stabilization plays an important role in recognizing objects. First, it prevents the object from slipping and facilitates the exploration of the object. Second, reaching a stable and repeatable position adds robustness to the learning algorithm and increases invariance with respect to the way in which the robot grasps the object. The stable poses are estimated using a Gaussian mixture model (GMM). We present experimental results showing that using our method the classifier can successfully distinguish 30 objects.We also compare our method with a benchmark experiment, in which the grasp stabilization is disabled. We show, with statistical significance, that our method outperforms the benchmark method.
Beyond Moore-Penrose Part II: The Sparse Pseudoinverse
Venue: arXiv cs.IT 2017
Year: 2017

This is the second part of a two-paper series on generalized inverses that minimize matrix norms. In Part II we focus on generalized inverses that are minimizers of entrywise p norms whose main representative is the sparse pseudoinverse for $p = 1$. We are motivated by the idea to replace the Moore-Penrose pseudoinverse by a sparser generalized inverse which is in some sense well-behaved. Sparsity implies that it is faster to apply the resulting matrix; well-behavedness would imply that we do not lose much in stability with respect to the least-squares performance of the MPP. We first address questions of uniqueness and non-zero count of (putative) sparse pseu-doinverses. We show that a sparse pseudoinverse is generically unique, and that it indeed reaches optimal sparsity for almost all matrices. We then turn to proving our main stability result: finite-size concentration bounds for the Frobenius norm of p-minimal inverses for $1 $\le$ p $\le$ 2$. Our proof is based on tools from convex analysis and random matrix theory, in particular the recently developed convex Gaussian min-max theorem. Along the way we prove several results about sparse representations and convex programming that were known folklore, but of which we could find no proof.
PSK Precoding in Multi-User MISO Systems
Venue: arXiv cs.IT 2017
Year: 2017

We consider the downlink scenario of multiuser multiple-input-single-output (MU-MISO) communication systems with constant envelope (CE) signals emitted from each antenna. This results in energy efficient power amplifiers (PAs). We propose a holistic CE precoding scheme based on the symbol-wise minimum squared error (SMSE) criterion. Additionally, we analyze the distortions introduced by low-resolution quantization to PSK for higher energy efficiency reasons. We present three solution algorithms and examine their performance to decide for the best pick for different quantization resolutions. Our results show that good performance can be achieved with minimal loss compared to an ideal unquantized case. Finally, we analyze and discuss the results and consider the overall complexity of the precoder as well as implementation issues.
Minimum BER Precoding in 1-Bit Massive MIMO Systems
Venue: arXiv cs.IT 2017
Year: 2017

1-bit digital-to-analog (DACs) and analog-to-digital converters (ADCs) are gaining more interest in massive MIMO systems for economical and computational efficiency. We present a new precoding technique to mitigate the inter-user-interference (IUI) and the channel distortions in a 1-bit downlink MUMISO system with QPSK symbols. The transmit signal vector is optimized taking into account the 1-bit quantization. We develop a sort of mapping based on a look-up table (LUT) between the input signal and the transmit signal. The LUT is updated for each channel realization. Simulation results show a significant gain in terms of the uncoded bit-error-ratio (BER) compared to the existing linear precoding techniques.
MMSE precoder for massive MIMO using 1-bit quantization
Venue: arXiv cs.IT 2017
Year: 2017

We propose a novel linear minimum-mean-squared-error (MMSE) precoder design for a downlink (DL) massive multiple-input-multiple-output (MIMO) scenario. For economical and computational efficiency reasons low resolution 1-bit digital-to-analog (DAC) and analog-to-digital (ADC) converters are used. This comes at the cost of performance gain that can be recovered by the large number of antennas deployed at the base station (BS) and an appropiate precoder design to mitigate the distortions due to the coarse quantization. The proposed precoder takes the quantization non-linearities into account and is split into a digital precoder and an analog precoder. We formulate the two-stage precoding problem such that the MSE of the users is minimized under the 1-bit constraint. In the simulations, we compare the new optimized precoding scheme with previously proposed linear precoders in terms of uncoded bit error ratio (BER).
DE-PACRR: Exploring Layers Inside the PACRR Model
Venue: arXiv cs.IR 2017
Year: 2017

Recent neural IR models have demonstrated deep learning's utility in ad-hoc information retrieval. However, deep models have a reputation for being black boxes, and the roles of a neural IR model's components may not be obvious at first glance. In this work, we attempt to shed light on the inner workings of a recently proposed neural IR model, namely the PACRR model, by visualizing the output of intermediate layers and by investigating the relationship between intermediate weights and the ultimate relevance score produced. We highlight several insights, hoping that such insights will be generally applicable.