OpenRaiser/Intern-Atlas huggingface dataset:
Neural Networks for Text Correction and Completion in Keyboard Decoding
Venue: arXiv cs.CL 2017
Year: 2017

Despite the ubiquity of mobile and wearable text messaging applications, the problem of keyboard text decoding is not tackled sufficiently in the light of the enormous success of the deep learning Recurrent Neural Network (RNN) and Convolutional Neural Networks (CNN) for natural language understanding. In particular, considering that the keyboard decoders should operate on devices with memory and processor resource constraints, makes it challenging to deploy industrial scale deep neural network (DNN) models. This paper proposes a sequence-to-sequence neural attention network system for automatic text correction and completion. Given an erroneous sequence, our model encodes character level hidden representations and then decodes the revised sequence thus enabling auto-correction and completion. We achieve this by a combination of character level CNN and gated recurrent unit (GRU) encoder along with and a word level gated recurrent unit (GRU) attention decoder. Unlike traditional language models that learn from billions of words, our corpus size is only 12 million words; an order of magnitude smaller. The memory footprint of our learnt model for inference and prediction is also an order of magnitude smaller than the conventional language model based text decoders. We report baseline performance for neural keyboard decoders in such limited domain. Our models achieve a word level accuracy of $90\%$ and a character error rate CER of $2.4\%$ over the Twitter typo dataset. We present a novel dataset of noisy to corrected mappings by inducing the noise distribution from the Twitter data over the OpenSubtitles 2009 dataset; on which our model predicts with a word level accuracy of $98\%$ and sequence accuracy of $68.9\%$. In our user study, our model achieved an average CER of $2.6\%$ with the state-of-the-art non-neural touch-screen keyboard decoder at CER of $1.6\%$.
Scalable Support Vector Clustering Using Budget
Venue: arXiv cs.LG 2017
Year: 2017

Owing to its application in solving the difficult and diverse clustering or outlier detection problem, support-based clustering has recently drawn plenty of attention. Support-based clustering method always undergoes two phases: finding the domain of novelty and performing clustering assignment. To find the domain of novelty, the training time given by the current solvers is typically over-quadratic in the training size, and hence precluding the usage of support-based clustering method for large-scale datasets. In this paper, we propose applying Stochastic Gradient Descent (SGD) framework to the first phase of support-based clustering for finding the domain of novelty and a new strategy to perform the clustering assignment. However, the direct application of SGD to the first phase of support-based clustering is vulnerable to the curse of kernelization, that is, the model size linearly grows up with the data size accumulated overtime. To address this issue, we invoke the budget approach which allows us to restrict the model size to a small budget. Our new strategy for clustering assignment enables a fast computation by means of reducing the task of clustering assignment on the full training set to the same task on a significantly smaller set. We also provide a rigorous theoretical analysis about the convergence rate for the proposed method. Finally, we validate our proposed method on the well-known datasets for clustering to show that the proposed method offers a comparable clustering quality while simultaneously achieving significant speedup in comparison with the baselines.
Taxonomy of Digital Forensics: Investigation Tools and Challenges
Venue: arXiv cs.CY 2017
Year: 2017

In today's world of computers, any kind of information can be made available within few clicks for different endeavors. The information may be tampered by changing the statistical properties and can be further used for criminal activities. These days, Cyber crimes are happening at a very large scale, and possess big threats to the security of an individual, firm, industry and even to developed countries. To combat such crimes, law enforcement agencies and investment institutions are incorporating supportive examination policies, procedures and protocols to address the complete investigation process. The paper entails a detailed review of several cyber crimes followed by various digital forensics processes involved in the cyber crime investigation. Further various digital forensics tools with detail explanation are discussed with their advantages, disadvantages, challenges, and drawbacks. A comparison among all the selected tools is also presented. Finally the paper recommends the need of training programs for the first res ponder and judgement of signature based image authentication.
Robust clustering of languages across Wikipedia growth
Venue: arXiv cs.DL 2017
Year: 2017

Wikipedia is the largest existing knowledge repository that is growing on a genuine crowdsourcing support. While the English Wikipedia is the most extensive and the most researched one with over five million articles, comparatively little is known about the behavior and growth of the remaining 283 smaller Wikipedias, the smallest of which, Afar, has only one article. Here we use a subset of this data, consisting of 14962 different articles, each of which exists in 26 different languages, from Arabic to Ukrainian. We study the growth of Wikipedias in these languages over a time span of 15 years. We show that, while an average article follows a random path from one language to another, there exist six well-defined clusters of Wikipedias that share common growth patterns. The make-up of these clusters is remarkably robust against the method used for their determination, as we verify via four different clustering methods. Interestingly, the identified Wikipedia clusters have little correlation with language families and groups. Rather, the growth of Wikipedia across different languages is governed by different factors, ranging from similarities in culture to information literacy.
Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges
Venue: arXiv cs.NI 2017
Year: 2017

While machine learning and artificial intelligence have long been applied in networking research, the bulk of such works has focused on supervised learning. Recently there has been a rising trend of employing unsupervised machine learning using unstructured raw network data to improve network performance and provide services such as traffic engineering, anomaly detection, Internet traffic classification, and quality of service optimization. The interest in applying unsupervised learning techniques in networking emerges from their great success in other fields such as computer vision, natural language processing, speech recognition, and optimal control (e.g., for developing autonomous self-driving cars). Unsupervised learning is interesting since it can unconstrain us from the need of labeled data and manual handcrafted feature engineering thereby facilitating flexible, general, and automated methods of machine learning. The focus of this survey paper is to provide an overview of the applications of unsupervised learning in the domain of networking. We provide a comprehensive survey highlighting the recent advancements in unsupervised learning techniques and describe their applications for various learning tasks in the context of networking. We also provide a discussion on future directions and open research issues, while also identifying potential pitfalls. While a few survey papers focusing on the applications of machine learning in networking have previously been published, a survey of similar scope and breadth is missing in literature. Through this paper, we advance the state of knowledge by carefully synthesizing the insights from these survey papers while also providing contemporary coverage of recent advances.
Why PairDiff works? -- A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection
Venue: arXiv cs.CL 2017
Year: 2017

Representing the semantic relations that exist between two given words (or entities) is an important first step in a wide-range of NLP applications such as analogical reasoning, knowledge base completion and relational information retrieval. A simple, yet surprisingly accurate method for representing a relation between two words is to compute the vector offset (\PairDiff) between their corresponding word embeddings. Despite the empirical success, it remains unclear as to whether \PairDiff is the best operator for obtaining a relational representation from word embeddings. We conduct a theoretical analysis of generalised bilinear operators that can be used to measure the $\ell_{2}$ relational distance between two word-pairs. We show that, if the word embeddings are standardised and uncorrelated, such an operator will be independent of bilinear terms, and can be simplified to a linear form, where \PairDiff is a special case. For numerous word embedding types, we empirically verify the uncorrelation assumption, demonstrating the general applicability of our theoretical result. Moreover, we experimentally discover \PairDiff from the bilinear relation composition operator on several benchmark analogy datasets.
VCExplorer: A Interactive Graph Exploration Framework Based on Hub Vertices with Graph Consolidation
Venue: arXiv cs.DB 2017
Year: 2017

Graphs have been widely used to model different information networks, such as the Web, biological networks and social networks (e.g. Twitter). Due to the size and complexity of these graphs, how to explore and utilize these graphs has become a very challenging problem. In this paper, we propose, VCExplorer, a new interactive graph exploration framework that integrates the strengths of graph visualization and graph summarization. Unlike existing graph visualization tools where vertices of a graph may be clustered into a smaller collection of super/virtual vertices, VCExplorer displays a small number of actual source graph vertices (called hubs) and summaries of the information between these vertices. We refer to such a graph as a HA-graph (Hub-based Aggregation Graph). This allows users to appreciate the relationship between the hubs, rather than super/virtual vertices. Users can navigate through the HA- graph by "drilling down" into the summaries between hubs to display more hubs. We illustrate how the graph aggregation techniques can be integrated into the exploring framework as the consolidated information to users. In addition, we propose efficient graph aggregation algorithms over multiple subgraphs via computation sharing. Extensive experimental evaluations have been conducted using both real and synthetic datasets and the results indicate the effectiveness and efficiency of VCExplorer for exploration.
Temporal Pattern Mining from Evolving Networks
Venue: arXiv cs.AI 2017
Year: 2017

Recently, evolving networks are becoming a suitable form to model many real-world complex systems, due to their peculiarities to represent the systems and their constituting entities, the interactions between the entities and the time-variability of their structure and properties. Designing computational models able to analyze evolving networks becomes relevant in many applications. The goal of this research project is to evaluate the possible contribution of temporal pattern mining techniques in the analysis of evolving networks. In particular, we aim at exploiting available snapshots for the recognition of valuable and potentially useful knowledge about the temporal dynamics exhibited by the network over the time, without making any prior assumption about the underlying evolutionary schema. Pattern-based approaches of temporal pattern mining can be exploited to detect and characterize changes exhibited by a network over the time, starting from observed snapshots.
On Energy Efficient Uplink Multi-User MIMO with Shared LNA Control
Venue: arXiv cs.IT 2017
Year: 2017

Implementation cost and power consumption are two important considerations in modern wireless communications, particularly in large-scale multi-antenna systems where the number of individual radio-frequency (RF) chains may be significantly larger than before. In this work, we propose to deploy a single low-noise amplifier (LNA) on the uplink multiple-input-multiple-output (MIMO) receiver to cover all antennas. This architecture, although favorable from the perspective of cost and power consumption, introduces challenges in the LNA gain control and user transmit power control. We formulate an energy efficiency maximization problem under practical system constraints, and prove that it is a constrained quasi-concave optimization problem. We then propose an efficient algorithm, Bisection -- Gradient Assisted Interior Point (B-GAIP), that solves this optimization problem. The optimality, convergence and complexity of B-GAIP are analyzed, and further corroborated via numerical simulations. In particular, the performance loss due to using a shared LNA as opposed to separate LNAs in each RF chain, when using B-GAIP to determine the LNA gain and user transmit power, is very small in both centralized and distributed MIMO systems.
On the Timed Temporal Logic Planning of Coupled Multi-Agent Systems
Venue: arXiv cs.SY 2017
Year: 2017

This paper presents a fully automated procedure for controller synthesis for multi-agent systems under coupling constraints. Each agent is modeled with dynamics consisting of two terms: the first one models the coupling constraints and the other one is an additional bounded control input. We aim to design these inputs so that each agent meets an individual high-level specification given as a Metric Interval Temporal Logic (MITL). First, a decentralized abstraction that provides a space and time discretization of the multi-agent system is designed. Second, by utilizing this abstraction and techniques from formal verification, we propose an algorithm that computes the individual runs which provably satisfy the high-level tasks. The overall approach is demonstrated in a simulation example conducted in MATLAB environment.
Contact Force and Joint Torque Estimation Using Skin
Venue: arXiv cs.RO 2017
Year: 2017

In this paper, we present algorithms to estimate external contact forces and joint torques using only skin, i.e. distributed tactile sensors. To deal with gaps between the tactile sensors (taxels), we use interpolation techniques. The application of these interpolation techniques allows us to estimate contact forces and joint torques without the need for expensive force-torque sensors. Validation was performed using the iCub humanoid robot.
Doctoral Advisor or Medical Condition: Towards Entity-specific Rankings of Knowledge Base Properties [Extended Version]
Venue: arXiv cs.IR 2017
Year: 2017

In knowledge bases such as Wikidata, it is possible to assert a large set of properties for entities, ranging from generic ones such as name and place of birth to highly profession-specific or background-specific ones such as doctoral advisor or medical condition. Determining a preference or ranking in this large set is a challenge in tasks such as prioritisation of edits or natural-language generation. Most previous approaches to ranking knowledge base properties are purely data-driven, that is, as we show, mistake frequency for interestingness. In this work, we have developed a human-annotated dataset of 350 preference judgments among pairs of knowledge base properties for fixed entities. From this set, we isolate a subset of pairs for which humans show a high level of agreement (87.5% on average). We show, however, that baseline and state-of-the-art techniques achieve only 61.3% precision in predicting human preferences for this subset. We then analyze what contributes to one property being rated as more important than another one, and identify that at least three factors play a role, namely (i) general frequency, (ii) applicability to similar entities and (iii) semantic similarity between property and entity. We experimentally analyze the contribution of each factor and show that a combination of techniques addressing all the three factors achieves 74% precision on the task. The dataset is available at www.kaggle.com/srazniewski/wikidatapropertyranking.
Plugo: a VLC Systematic Perspective of Large-scale Indoor Localization
Venue: arXiv cs.NI 2017
Year: 2017

Indoor localization based on Visible Light Communication (VLC) has been in favor with both the academia and industry for years. In this paper, we present a prototyping photodiode-based VLC system towards large-scale localization. Specially, we give in-depth analysis of the design constraints and considerations for large-scale indoor localization research. After that we identify the key enablers for such systems: 1) distributed architecture, 2) one-way communication, and 3) random multiple access. Accordingly, we propose Plugo -- a photodiode-based VLC system conforming to the aforementioned criteria. We present a compact design of the VLC-compatible LED bulbs featuring plug-and-go use-cases. The basic framed slotted Additive Links On-line Hawaii Area (ALOHA) is exploited to achieve random multiple access over the shared optical medium. We show its effectiveness in beacon broadcasting by experiments, and further demonstrate its scalability to large-scale scenarios through simulations. Finally, preliminary localization experiments are conducted using fingerprinting-based methods in a customized testbed, achieving an average accuracy of 0.14 m along with a 90-percentile accuracy of 0.33 m.
Survey on Semi-Explicit Time Integration of Eddy Current Problems
Venue: arXiv cs.CE 2017
Year: 2017

The spatial discretization of the magnetic vector potential formulation of magnetoquasistatic field problems results in an infinitely stiff differential-algebraic equation system. It is transformed into a finitely stiff ordinary differential equation system by applying a generalized Schur complement. Applying the explicit Euler time integration scheme to this system results in a small maximum stable time step size. Fast computations are required in every time step to yield an acceptable overall simulation time. Several acceleration methods are presented.
On Compiling DNNFs without Determinism
Venue: arXiv cs.AI 2017
Year: 2017

State-of-the-art knowledge compilers generate deterministic subsets of DNNF, which have been recently shown to be exponentially less succinct than DNNF. In this paper, we propose a new method to compile DNNFs without enforcing determinism necessarily. Our approach is based on compiling deterministic DNNFs with the addition of auxiliary variables to the input formula. These variables are then existentially quantified from the deterministic structure in linear time, which would lead to a DNNF that is equivalent to the input formula and not necessarily deterministic. On the theoretical side, we show that the new method could generate exponentially smaller DNNFs than deterministic ones, even by adding a single auxiliary variable. Further, we show that various existing techniques that introduce auxiliary variables to the input formulas can be employed in our framework. On the practical side, we empirically demonstrate that our new method can significantly advance DNNF compilation on certain benchmarks.
Practical Machine Learning for Cloud Intrusion Detection: Challenges and the Way Forward
Venue: arXiv cs.CR 2017
Year: 2017

Operationalizing machine learning based security detections is extremely challenging, especially in a continuously evolving cloud environment. Conventional anomaly detection does not produce satisfactory results for analysts that are investigating security incidents in the cloud. Model evaluation alone presents its own set of problems due to a lack of benchmark datasets. When deploying these detections, we must deal with model compliance, localization, and data silo issues, among many others. We pose the problem of "attack disruption" as a way forward in the security data science space. In this paper, we describe the framework, challenges, and open questions surrounding the successful operationalization of machine learning based security detections in a cloud environment and provide some insights on how we have addressed them.
Discrete-Time Polar Opinion Dynamics with Susceptibility
Venue: arXiv cs.SI 2017
Year: 2017

This paper considers a discrete-time opinion dynamics model in which each individual's susceptibility to being influenced by others is dependent on her current opinion. We assume that the social network has time-varying topology and that the opinions are scalars on a continuous interval. We first propose a general opinion dynamics model based on the DeGroot model, with a general function to describe the functional dependence of each individual's susceptibility on her own opinion, and show that this general model is analogous to the Friedkin-Johnsen model, which assumes a constant susceptibility for each individual. We then consider two specific functions in which the individual's susceptibility depends on the \emph{polarity} of her opinion, and provide motivating social examples. First, we consider stubborn positives, who have reduced susceptibility if their opinions are at one end of the interval and increased susceptibility if their opinions are at the opposite end. A court jury is used as a motivating example. Second, we consider stubborn neutrals, who have reduced susceptibility when their opinions are in the middle of the spectrum, and our motivating examples are social networks discussing established social norms or institutionalized behavior. For each specific susceptibility model, we establish the initial and graph topology conditions in which consensus is reached, and develop necessary and sufficient conditions on the initial conditions for the final consensus value to be at either extreme of the opinion interval. Simulations are provided to show the effects of the susceptibility function when compared to the DeGroot model.
Convolutional neural networks that teach microscopes how to image
Venue: arXiv cs.CV 2017
Year: 2017

Deep learning algorithms offer a powerful means to automatically analyze the content of medical images. However, many biological samples of interest are primarily transparent to visible light and contain features that are difficult to resolve with a standard optical microscope. Here, we use a convolutional neural network (CNN) not only to classify images, but also to optimize the physical layout of the imaging device itself. We increase the classification accuracy of a microscope's recorded images by merging an optical model of image formation into the pipeline of a CNN. The resulting network simultaneously determines an ideal illumination arrangement to highlight important sample features during image acquisition, along with a set of convolutional weights to classify the detected images post-capture. We demonstrate our joint optimization technique with an experimental microscope configuration that automatically identifies malaria-infected cells with 5-10% higher accuracy than standard and alternative microscope lighting designs.
Sorting with Recurrent Comparison Errors
Venue: arXiv cs.DS 2017
Year: 2017

We present a sorting algorithm for the case of recurrent random comparison errors. The algorithm essentially achieves simultaneously good properties of previous algorithms for sorting $n$ distinct elements in this model. In particular, it runs in $O(n^2)$ time, the maximum dislocation of the elements in the output is $O(\log n)$, while the total dislocation is $O(n)$. These guarantees are the best possible since we prove that even randomized algorithms cannot achieve $o(\log n)$ maximum dislocation with high probability, or $o(n)$ total dislocation in expectation, regardless of their running time.