OpenRaiser/Intern-Atlas huggingface dataset:
Predicting User Roles in Social Networks using Transfer Learning with Feature Transformation
Venue: arXiv cs.SI 2016
Year: 2016

How can we recognise social roles of people, given a completely unlabelled social network? We present a transfer learning approach to network role classification based on feature transformations from each network's local feature distribution to a global feature space. Experiments are carried out on real-world datasets. (See manuscript for the full abstract.)
Increasing the throughput of machine translation systems using clouds
Venue: arXiv cs.CL 2016
Year: 2016

The manuscript presents an experiment at implementation of a Machine Translation system in a MapReduce model. The empirical evaluation was done using fully implemented translation systems embedded into the MapReduce programming model. Two machine translation paradigms were studied: shallow transfer Rule Based Machine Translation and Statistical Machine Translation. The results show that the MapReduce model can be successfully used to increase the throughput of a machine translation system. Furthermore this method enhances the throughput of a machine translation system without decreasing the quality of the translation output. Thus, the present manuscript also represents a contribution to the seminal work in natural language processing, specifically Machine Translation. It first points toward the importance of the definition of the metric of throughput of translation system and, second, the applicability of the machine translation task to the MapReduce paradigm.
Distant supervision for emotion detection using Facebook reactions
Venue: arXiv cs.CL 2016
Year: 2016

We exploit the Facebook reaction feature in a distant supervised fashion to train a support vector machine classifier for emotion detection, using several feature combinations and combining different Facebook pages. We test our models on existing benchmarks for emotion detection and show that employing only information that is derived completely automatically, thus without relying on any handcrafted lexicon as it's usually done, we can achieve competitive results. The results also show that there is large room for improvement, especially by gearing the collection of Facebook pages, with a view to the target domain.
Access Control in Linked Data Using WebID
Venue: arXiv cs.CR 2016
Year: 2016

Linked Data technologies become increasingly important in many domains. Key factors for their breakthrough are security and trust, especially when sensible or personal data are involved. Classical means for access control lack granularity when parts of the Linked Data graph must be protected. The WebID, combining semantic web concepts with methods from certificate based authentication and authorization, seems promising to fulfill all requirements concerning security and trust in the semantic web. In the context of the PerSemID project, we challenged the WebID technology in a practical scenario coming from the domain of lifelong learning and student mobility. In our use case of study enrollment, we use WebIDs for authentication and to grant access to parts of the triple stores of the different stakeholders. Cross domain triple store interactions are used to exchange data between the involved parties. Our fully implemented PoC exemplifies an application built on Linked Data and WebID and allows us to judge the usability and security of WebID technology in a real world scenario.
Applying Bag of System Calls for Anomalous Behavior Detection of Applications in Linux Containers
Venue: arXiv cs.CR 2016
Year: 2016

In this paper, we present the results of using bags of system calls for learning the behavior of Linux containers for use in anomaly-detection based intrusion detection system. By using system calls of the containers monitored from the host kernel for anomaly detection, the system does not require any prior knowledge of the container nature, neither does it require altering the container or the host kernel.
Incremental Sequence Learning
Venue: arXiv cs.LG 2016
Year: 2016

Deep learning research over the past years has shown that by increasing the scope or difficulty of the learning problem over time, increasingly complex learning problems can be addressed. We study incremental learning in the context of sequence learning, using generative RNNs in the form of multi-layer recurrent Mixture Density Networks. While the potential of incremental or curriculum learning to enhance learning is known, indiscriminate application of the principle does not necessarily lead to improvement, and it is essential therefore to know which forms of incremental or curriculum learning have a positive effect. This research contributes to that aim by comparing three instantiations of incremental or curriculum learning. We introduce Incremental Sequence Learning, a simple incremental approach to sequence learning. Incremental Sequence Learning starts out by using only the first few steps of each sequence as training data. Each time a performance criterion has been reached, the length of the parts of the sequences used for training is increased. We introduce and make available a novel sequence learning task and data set: predicting and classifying MNIST pen stroke sequences. We find that Incremental Sequence Learning greatly speeds up sequence learning and reaches the best test performance level of regular sequence learning 20 times faster, reduces the test error by 74%, and in general performs more robustly; it displays lower variance and achieves sustained progress after all three comparison methods have stopped improving. The other instantiations of curriculum learning do not result in any noticeable improvement. A trained sequence prediction model is also used in transfer learning to the task of sequence classification, where it is found that transfer learning realizes improved classification performance compared to methods that learn to classify from scratch.
A Modular Theory of Feature Learning
Venue: arXiv cs.LG 2016
Year: 2016

Learning representations of data, and in particular learning features for a subsequent prediction task, has been a fruitful area of research delivering impressive empirical results in recent years. However, relatively little is understood about what makes a representation `good'. We propose the idea of a risk gap induced by representation learning for a given prediction context, which measures the difference in the risk of some learner using the learned features as compared to the original inputs. We describe a set of sufficient conditions for unsupervised representation learning to provide a benefit, as measured by this risk gap. These conditions decompose the problem of when representation learning works into its constituent parts, which can be separately evaluated using an unlabeled sample, suitable domain-specific assumptions about the joint distribution, and analysis of the feature learner and subsequent supervised learner. We provide two examples of such conditions in the context of specific properties of the unlabeled distribution, namely when the data lies close to a low-dimensional manifold and when it forms clusters. We compare our approach to a recently proposed analysis of semi-supervised learning.
Low Data Drug Discovery with One-shot Learning
Venue: arXiv cs.LG 2016
Year: 2016

Recent advances in machine learning have made significant contributions to drug discovery. Deep neural networks in particular have been demonstrated to provide significant boosts in predictive power when inferring the properties and activities of small-molecule compounds. However, the applicability of these techniques has been limited by the requirement for large amounts of training data. In this work, we demonstrate how one-shot learning can be used to significantly lower the amounts of data required to make meaningful predictions in drug discovery applications. We introduce a new architecture, the residual LSTM embedding, that, when combined with graph convolutional neural networks, significantly improves the ability to learn meaningful distance metrics over small-molecules. We open source all models introduced in this work as part of DeepChem, an open-source framework for deep-learning in drug discovery.
Length Matters: Clustering System Log Messages using Length of Words
Venue: arXiv cs.OH 2016
Year: 2016

The analysis techniques of system log messages (syslog messages) have a long history from when the syslog mechanism was invented. Typically, the analysis consists of two parts, one is a message template generation, and the other is finding something interesting using the messages classified by the inferred templates. It is important to generate better templates to achieve better, precise, or convincible analysis results. In this paper, we propose a classification methodology using the length of words of each message. Our method is suitable for online template generation because it does not require two-pass analysis to generate template messages, that is an important factor considering increasing amount of log messages produced by a large number of system components such as cloud infrastructure.
Getting Started with Neural Models for Semantic Matching in Web Search
Venue: arXiv cs.IR 2016
Year: 2016

The vocabulary mismatch problem is a long-standing problem in information retrieval. Semantic matching holds the promise of solving the problem. Recent advances in language technology have given rise to unsupervised neural models for learning representations of words as well as bigger textual units. Such representations enable powerful semantic matching methods. This survey is meant as an introduction to the use of neural models for semantic matching. To remain focused we limit ourselves to web search. We detail the required background and terminology, a taxonomy grouping the rapidly growing body of work in the area, and then survey work on neural models for semantic matching in the context of three tasks: query suggestion, ad retrieval, and document retrieval. We include a section on resources and best practices that we believe will help readers who are new to the area. We conclude with an assessment of the state-of-the-art and suggestions for future work.
Privacy of the Internet of Things: A Systematic Literature Review (Extended Discussion)
Venue: arXiv cs.CY 2016
Year: 2016

The Internet of Things' potential for major privacy invasion is a concern. This paper reports on a systematic literature review of privacy-preserving solutions appearing in the research literature and in the media. We analysed proposed solutions in terms of the techniques they deployed and the extent to which they satisfied core privacy principles. We found that very few solutions satisfied all core privacy principles. We also identified a number of key knowledge gaps in the course of the analysis. In particular, we found that most solution providers assumed that end users would be willing to expend effort to preserve their privacy; that they would be motivated to act to preserve their privacy. The validity of this assumption needs to be proved, since it cannot simply be assumed that people would necessarily be willing to engage with these solutions. We suggest this as a topic for future research.
Evaluation of spatial trees for simulation of biological tissue
Venue: arXiv cs.CG 2016
Year: 2016

Spatial organization is a core challenge for all large agent-based models with local interactions. In biological tissue models, spatial search and reinsertion are frequently reported as the most expensive steps of the simulation. One of the main methods utilized in order to maintain both favorable algorithmic complexity and accuracy is spatial hierarchies. In this paper, we seek to clarify to which extent the choice of spatial tree affects performance, and also to identify which spatial tree families are optimal for such scenarios. We make use of a prototype of the new BioDynaMo tissue simulator for evaluating performances as well as for the implementation of the characteristics of several different trees.
Efficient Summarization with Read-Again and Copy Mechanism
Venue: arXiv cs.CL 2016
Year: 2016

Encoder-decoder models have been widely used to solve sequence to sequence prediction tasks. However current approaches suffer from two shortcomings. First, the encoders compute a representation of each word taking into account only the history of the words it has read so far, yielding suboptimal representations. Second, current decoders utilize large vocabularies in order to minimize the problem of unknown words, resulting in slow decoding times. In this paper we address both shortcomings. Towards this goal, we first introduce a simple mechanism that first reads the input sequence before committing to a representation of each word. Furthermore, we propose a simple copy mechanism that is able to exploit very small vocabularies and handle out-of-vocabulary words. We demonstrate the effectiveness of our approach on the Gigaword dataset and DUC competition outperforming the state-of-the-art.
Disentangling factors of variation in deep representations using adversarial training
Venue: arXiv cs.LG 2016
Year: 2016

We introduce a conditional generative model for learning to disentangle the hidden factors of variation within a set of labeled observations, and separate them into complementary codes. One code summarizes the specified factors of variation associated with the labels. The other summarizes the remaining unspecified variability. During training, the only available source of supervision comes from our ability to distinguish among different observations belonging to the same class. Examples of such observations include images of a set of labeled objects captured at different viewpoints, or recordings of set of speakers dictating multiple phrases. In both instances, the intra-class diversity is the source of the unspecified factors of variation: each object is observed at multiple viewpoints, and each speaker dictates multiple phrases. Learning to disentangle the specified factors from the unspecified ones becomes easier when strong supervision is possible. Suppose that during training, we have access to pairs of images, where each pair shows two different objects captured from the same viewpoint. This source of alignment allows us to solve our task using existing methods. However, labels for the unspecified factors are usually unavailable in realistic scenarios where data acquisition is not strictly controlled. We address the problem of disentanglement in this more general setting by combining deep convolutional autoencoders with a form of adversarial training. Both factors of variation are implicitly captured in the organization of the learned embedding space, and can be used for solving single-image analogies. Experimental results on synthetic and real datasets show that the proposed method is capable of generalizing to unseen classes and intra-class variabilities.
On Service-Chaining Strategies using Virtual Network Functions in Operator Networks
Venue: arXiv cs.NI 2016
Year: 2016

Network functions (e.g., firewalls, load balancers, etc.) have been traditionally provided through proprietary hardware appliances. Often, hardware appliances need to be hardwired back to back to form a service chain providing chained network functions. Hardware appliances cannot be provisioned on demand since they are statically embedded in the network topology, making creation, insertion, modification, upgrade, and removal of service chains complex, and also slowing down service innovation. Hence, network operators are starting to deploy Virtual Network Functions (VNFs), which are virtualized over commodity hardware. VNFs can be deployed in Data Centers (DCs) or in Network Function Virtualization (NFV) capable network elements (nodes) such as routers and switches. NFV capable nodes and DCs together form a Network enabled Cloud (NeC) that helps to facilitate the dynamic service chaining required to support evolving network traffic and its service demands. In this study, we focus on the VNF service chain placement and traffic routing problem, and build a model for placing a VNF service chain while minimizing network resource consumption. Our results indicate that a NeC having a DC and NFV capable nodes can significantly reduce network-resource consumption.
Deep Recurrent Neural Network for Mobile Human Activity Recognition with High Throughput
Venue: arXiv cs.CV 2016
Year: 2016

In this paper, we propose a method of human activity recognition with high throughput from raw accelerometer data applying a deep recurrent neural network (DRNN), and investigate various architectures and its combination to find the best parameter values. The "high throughput" refers to short time at a time of recognition. We investigated various parameters and architectures of the DRNN by using the training dataset of 432 trials with 6 activity classes from 7 people. The maximum recognition rate was 95.42% and 83.43% against the test data of 108 segmented trials each of which has single activity class and 18 multiple sequential trials, respectively. Here, the maximum recognition rates by traditional methods were 71.65% and 54.97% for each. In addition, the efficiency of the found parameters was evaluated by using additional dataset. Further, as for throughput of the recognition per unit time, the constructed DRNN was requiring only 1.347 [ms], while the best traditional method required 11.031 [ms] which includes 11.027 [ms] for feature calculation. These advantages are caused by the compact and small architecture of the constructed real time oriented DRNN.
Smartphone-based Vehicle Telematics - A Ten-Year Anniversary
Venue: arXiv cs.CY 2016
Year: 2016

Just like it has irrevocably reshaped social life, the fast growth of smartphone ownership is now beginning to revolutionize the driving experience and change how we think about automotive insurance, vehicle safety systems, and traffic research. This paper summarizes the first ten years of research in smartphone-based vehicle telematics, with a focus on user-friendly implementations and the challenges that arise due to the mobility of the smartphone. Notable academic and industrial projects are reviewed, and system aspects related to sensors, energy consumption, cloud computing, vehicular ad hoc networks, and human-machine interfaces are examined. Moreover, we highlight the differences between traditional and smartphonebased automotive navigation, and survey the state-of-the-art in smartphone-based transportation mode classification, driver classification, and road condition monitoring. Future advances are expected to be driven by improvements in sensor technology, evidence of the societal benefits of current implementations, and the establishment of industry standards for sensor fusion and driver assessment
Improving Belief Propagation Decoding of Polar Codes Using Scattered EXIT Charts
Venue: arXiv cs.IT 2016
Year: 2016

For finite length polar codes, channel polarization leaves a significant number of channels not fully polarized. Adding a Cyclic Redundancy Check (CRC) to better protect information on the semi-polarized channels has already been successfully applied in the literature, and is straightforward to be used in combination with Successive Cancellation List (SCL) decoding. Belief Propagation (BP) decoding, however, offers more potential for exploiting parallelism in hardware implementation, and thus, we focus our attention on improving the BP decoder. Specifically, similar to the CRC strategy in the SCL-case, we use a short-length "auxiliary" LDPC code together with the polar code to provide a significant improvement in terms of BER. We present the novel concept of "scattered" EXIT charts to design such auxiliary LDPC codes, and achieve net coding gains (Le. for the same total rate) of 0.4 dB at BER of 1E-5 compared to the conventional BP decoder.
Using text mining and machine learning for detection of child abuse
Venue: arXiv cs.CY 2016
Year: 2016

Abuse in any form is a grave threat to a child's health. Public health institutions in the Netherlands try to identify and prevent different kinds of abuse, and building a decision support system can help such institutions achieve this goal. Such decision support relies on the analysis of relevant child health data. A significant part of the medical data that the institutions have on children is unstructured, and in the form of free text notes. In this research, we employ machine learning and text mining techniques to detect patterns of possible child abuse in the data. The resulting model achieves a high score in classifying cases of possible abuse. We then describe our implementation of the decision support API at a municipality in the Netherlands.
Design and Analysis of Compressive Antenna Arrays for Direction of Arrival Estimation
Venue: arXiv cs.IT 2016
Year: 2016

In this paper we investigate the design of compressive antenna arrays for direction of arrival (DOA) estimation that aim to provide a larger aperture with a reduced hardware complexity by a linear combination of the antenna outputs to a lower number of receiver channels. We present a basic receiver architecture of such a compressive array and introduce a generic system model that includes different options for the hardware implementation. We then discuss the design of the analog combining network that performs the receiver channel reduction, and propose two design approaches. The first approach is based on the spatial correlation function which is a low-complexity scheme that in certain cases admits a closed-form solution. The second approach is based on minimizing the Cramer-Rao Bound (CRB) with the constraint to limit the probability of false detection of paths to a pre-specified level. Our numerical simulations demonstrate the superiority of the proposed optimized compressive arrays compared to the sparse arrays of the same complexity and to compressive arrays with randomly chosen combining kernels.
Computational Interpretations of Markov's principle
Venue: arXiv cs.LO 2016
Year: 2016

Markov's principle is a statement that originated in the Russian school of Constructive Mathematics and stated originally that "if it is impossible that an algorithm does not terminate, then it will terminate". This principle has been adapted to many different contexts, and in particular we are interested in its most common version for arithmetic, which can be stated as "given a total recursive function f , if it is impossible that there is no n for which f(n) = 0, then there exists an n such that f(n) = 0". This is in general not accepted in constructivism, where stating an existential statement requires one to be able to show at request a witness for the statement: here there is no clear way to choose such an n. We introduce more in detail the context of constructive mathematics from different points of view, and we show how they are related to Markov's principle. In particular, several realizability semantics are presented, which provide interpretations of logical systems by means of different computational concepts (mainly, recursive functions and lambda calculi). This field of research gave origin to the well known paradigm often called Curry-Howrd isomorphism, or also propositions as types, that states a correspondence between proofs in logic and programs in computer science. Thanks to this the field of proof theory, that is the metamathematical investigations of proofs as mathematical objects, became of interest for computer science and in particular for the study of programming languages. By using modern research on the Curry-Howard isomorphism, we will obtain a more refined interpretation of Markov's principle. We will then use this results to investigate the logical properties of systems related to the principle, and introduce a proof transformation technique to interpret constructively some non-constructive proofs of arithmetic.