OpenRaiser/Intern-Atlas huggingface dataset:
Trustworthy Configuration Management for Networked Devices using Distributed Ledgers
Venue: arXiv cs.CR 2018
Year: 2018

Numerous IoT applications, like building automation or process control of industrial sites, exist today. These applications inherently have a strong connection to the physical world. Hence, IT security threats cannot only cause problems like data leaks but also safety issues which might harm people. Attacks on IT systems are not only performed by outside attackers but also insiders like administrators. For this reason, we present ongoing work on a configuration management system (CMS) that provides control over administrators, restrains their rights, and enforces separation of concerns. We reach this goal by conducting a configuration management process that requires multi-party authorization for critical configurations to achieve Byzantine fault tolerance against attacks and faults by administrators. Only after a configuration has been authorized by multiple experts, it is applied to the targeted devices. For the whole configuration management process, our CMS guarantees accountability and traceability. Lastly, our system is tamper-resistant as we leverage Hyperledger Fabric, which provides a distributed execution environment for our CMS and a blockchain-based distributed ledger that we use to store the configurations. A beneficial side effect of this approach is that our CMS is also suitable to manage configurations for infrastructure shared across different organizations that do not need to trust each other.
μ-cuDNN: Accelerating Deep Learning Frameworks with Micro-Batching
Venue: arXiv cs.LG 2018
Year: 2018

NVIDIA cuDNN is a low-level library that provides GPU kernels frequently used in deep learning. Specifically, cuDNN implements several equivalent convolution algorithms, whose performance and memory footprint may vary considerably, depending on the layer dimensions. When an algorithm is automatically selected by cuDNN, the decision is performed on a per-layer basis, and thus it often resorts to slower algorithms that fit the workspace size constraints. We present μ-cuDNN, a transparent wrapper library for cuDNN, which divides layers' mini-batch computation into several micro-batches. Based on Dynamic Programming and Integer Linear Programming, μ-cuDNN enables faster algorithms by decreasing the workspace requirements. At the same time, μ-cuDNN keeps the computational semantics unchanged, so that it decouples statistical efficiency from the hardware efficiency safely. We demonstrate the effectiveness of μ-cuDNN over two frameworks, Caffe and TensorFlow, achieving speedups of 1.63x for AlexNet and 1.21x for ResNet-18 on P100-SXM2 GPU. These results indicate that using micro-batches can seamlessly increase the performance of deep learning, while maintaining the same memory footprint.
Group Anomaly Detection using Deep Generative Models
Venue: arXiv cs.CV 2018
Year: 2018

Unlike conventional anomaly detection research that focuses on point anomalies, our goal is to detect anomalous collections of individual data points. In particular, we perform group anomaly detection (GAD) with an emphasis on irregular group distributions (e.g. irregular mixtures of image pixels). GAD is an important task in detecting unusual and anomalous phenomena in real-world applications such as high energy particle physics, social media, and medical imaging. In this paper, we take a generative approach by proposing deep generative models: Adversarial autoencoder (AAE) and variational autoencoder (VAE) for group anomaly detection. Both AAE and VAE detect group anomalies using point-wise input data where group memberships are known a priori. We conduct extensive experiments to evaluate our models on real-world datasets. The empirical results demonstrate that our approach is effective and robust in detecting group anomalies.
Scalable and Interpretable One-class SVMs with Deep Learning and Random Fourier features
Venue: arXiv cs.LG 2018
Year: 2018

One-class support vector machine (OC-SVM) for a long time has been one of the most effective anomaly detection methods and extensively adopted in both research as well as industrial applications. The biggest issue for OC-SVM is yet the capability to operate with large and high-dimensional datasets due to optimization complexity. Those problems might be mitigated via dimensionality reduction techniques such as manifold learning or autoencoder. However, previous work often treats representation learning and anomaly prediction separately. In this paper, we propose autoencoder based one-class support vector machine (AE-1SVM) that brings OC-SVM, with the aid of random Fourier features to approximate the radial basis kernel, into deep learning context by combining it with a representation learning architecture and jointly exploit stochastic gradient descent to obtain end-to-end training. Interestingly, this also opens up the possible use of gradient-based attribution methods to explain the decision making for anomaly detection, which has ever been challenging as a result of the implicit mappings between the input space and the kernel space. To the best of our knowledge, this is the first work to study the interpretability of deep learning in anomaly detection. We evaluate our method on a wide range of unsupervised anomaly detection tasks in which our end-to-end training architecture achieves a performance significantly better than the previous work using separate training.
Constructive Analysis of S1S and Büchi Automata
Venue: arXiv cs.LO 2018
Year: 2018

We study S1S and Büchi automata in the constructive type theory of the Coq proof assistant. For UP semantics (ultimately periodic sequences), we verify Büchi's translation of formulas to automata and thereby establish decidability of S1S constructively. For AS semantics (all sequences), we verify Büchi's translation assuming that sequences over finite semigroups have Ramseyan factorisations (RF). Assuming RF, UP semantics and AS semantics agree. RF is a consequence of Ramsey's theorem and implies the infinite pigeonhole principle, which is known to be unprovable constructively. We show that each of the following properties holds for UP semantics but is equivalent to RF for AS semantics: excluded middle of formula satisfaction, excluded middle of automaton acceptance, and existence of complement automata.
Incorporating Dictionaries into Deep Neural Networks for the Chinese Clinical Named Entity Recognition
Venue: arXiv cs.CL 2018
Year: 2018

Clinical Named Entity Recognition (CNER) aims to identify and classify clinical terms such as diseases, symptoms, treatments, exams, and body parts in electronic health records, which is a fundamental and crucial task for clinical and translational research. In recent years, deep neural networks have achieved significant success in named entity recognition and many other Natural Language Processing (NLP) tasks. Most of these algorithms are trained end to end, and can automatically learn features from large scale labeled datasets. However, these data-driven methods typically lack the capability of processing rare or unseen entities. Previous statistical methods and feature engineering practice have demonstrated that human knowledge can provide valuable information for handling rare and unseen cases. In this paper, we address the problem by incorporating dictionaries into deep neural networks for the Chinese CNER task. Two different architectures that extend the Bi-directional Long Short-Term Memory (Bi-LSTM) neural network and five different feature representation schemes are proposed to handle the task. Computational results on the CCKS-2017 Task 2 benchmark dataset show that the proposed method achieves the highly competitive performance compared with the state-of-the-art deep learning methods.
Specifying, Monitoring, and Executing Workflows in Linked Data Environments
Venue: arXiv cs.AI 2018
Year: 2018

We present an ontology for representing workflows over components with Read-Write Linked Data interfaces and give an operational semantics to the ontology via a rule language. Workflow languages have been successfully applied for modelling behaviour in enterprise information systems, in which the data is often managed in a relational database. Linked Data interfaces have been widely deployed on the web to support data integration in very diverse domains, increasingly also in scenarios involving the Internet of Things, in which application behaviour is often specified using imperative programming languages. With our work we aim to combine workflow languages, which allow for the high-level specification of application behaviour by non-expert users, with Linked Data, which allows for decentralised data publication and integrated data access. We show that our ontology is expressive enough to cover the basic workflow patterns and demonstrate the applicability of our approach with a prototype system that observes pilots carrying out tasks in a mixed-reality aircraft cockpit. On a synthetic benchmark from the building automation domain, the runtime scales linearly with the size of the number of Internet of Things devices.
Incentive design for learning in user-recommendation systems with time-varying states
Venue: arXiv cs.GT 2018
Year: 2018

We consider the problem of how strategic users with asymmetric information can learn an underlying time varying state in a user-recommendation system. Users who observe private signals about the state, sequentially make a decision about buying a product whose value varies with time in an ergodic manner. We formulate the team problem as an instance of decentralized stochastic control problem and characterize its optimal policies. With strategic users, we design incentives such that users reveal their true private signals, so that the gap between the strategic and team objective is small and the overall expected incentive payments are also small.
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Venue: arXiv cs.CV 2018
Year: 2018

We address the problem of text-based activity retrieval in video. Given a sentence describing an activity, our task is to retrieve matching clips from an untrimmed video. To capture the inherent structures present in both text and video, we introduce a multilevel model that integrates vision and language features earlier and more tightly than prior work. First, we inject text features early on when generating clip proposals, to help eliminate unlikely clips and thus speed up processing and boost performance. Second, to learn a fine-grained similarity metric for retrieval, we use visual features to modulate the processing of query sentences at the word level in a recurrent neural network. A multi-task loss is also employed by adding query re-generation as an auxiliary task. Our approach significantly outperforms prior work on two challenging benchmarks: Charades-STA and ActivityNet Captions.
Coordinated Control of Energy Storage in Networked Microgrids under Unpredicted Load Demands
Venue: arXiv cs.SY 2018
Year: 2018

In this paper a nonlinear control design for power balancing in networked microgrids using energy storage devices is presented. Each microgrid is considered to be interfaced to the distribution feeder though a solid-state transformer (SST). The internal duty cycle based controllers of each SST ensures stable regulation of power commands during normal operation. But problem arises when a sudden change in load or generation occurs in any microgrid in a completely unpredicted way in between the time instants at which the SSTs receive their power setpoints. In such a case, the energy storage unit in that microgrid must produce or absorb the deficit power. The challenge lies in designing a suitable regulator for this purpose owing to the nonlinearity of the battery model and its coupling with the nonlinear SST dynamics. We design an input-output linearization based controller, and show that it guarantees closed-loop stability via a cascade connection with the SST model. The design is also extended to the case when multiple SSTs must coordinate their individual storage controllers to assist a given SST whose storage capacity is insufficient to serve the unpredicted load. The design is verified using the IEEE 34-bus distribution system with nine SST-driven microgrids.
Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection
Venue: arXiv cs.RO 2018
Year: 2018

To assure that an autonomous car is driving safely on public roads, its object detection module should not only work correctly, but show its prediction confidence as well. Previous object detectors driven by deep learning do not explicitly model uncertainties in the neural network. We tackle with this problem by presenting practical methods to capture uncertainties in a 3D vehicle detector for Lidar point clouds. The proposed probabilistic detector represents reliable epistemic uncertainty and aleatoric uncertainty in classification and localization tasks. Experimental results show that the epistemic uncertainty is related to the detection accuracy, whereas the aleatoric uncertainty is influenced by vehicle distance and occlusion. The results also show that we can improve the detection performance by 1%-5% by modeling the aleatoric uncertainty.
LiDAR and Camera Calibration using Motion Estimated by Sensor Fusion Odometry
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we propose a method of targetless and automatic Camera-LiDAR calibration. Our approach is an extension of hand-eye calibration framework to 2D-3D calibration. By using the sensor fusion odometry method, the scaled camera motions are calculated with high accuracy. In addition to this, we clarify the suitable motion for this calibration method. The proposed method only requires the three-dimensional point cloud and the camera image and does not need other information such as reflectance of LiDAR and to give initial extrinsic parameter. In the experiments, we demonstrate our method using several sensor configurations in indoor and outdoor scenes to verify the effectiveness. The accuracy of our method achieves more than other comparable state-of-the-art methods.
Area formation and content assignment for LTE broadcasting
Venue: arXiv cs.NI 2018
Year: 2018

Broadcasting and multicasting services in LTE networks are shaping up to be an effective way to provide popular content. A key requirement is that cells are aggregated into areas where a tight time synchronization among transmissions is enforced, so as to broadcast the same radio resources. Our paper addresses a facet of LTE broadcasting that has so far received little attention: the creation of broadcasting areas and the assignment of content to them in order to efficiently exploit radio resources and satisfy user requests. Our original clustering approach, named Single-Content Fusion, achieves these goals by initially aggregating cells into single-content areas and maximizing cell similarity in content interests. Aggregated areas are then merged into multiple-content areas by virtue of similar spatial coverage. We show the validity of our solution pointing out the advantages it provides in comparison to other approaches. We also discuss the impact of various system factors (e.g., number of served users, broadcast data rate, area size) and the scalability of our proposal in large, realistic scenarios with both static and time-varying user interest.
Robust Transmission Design for Multi-Cell D2D Underlaid Cellular Networks
Venue: arXiv cs.IT 2018
Year: 2018

This paper investigates the robust transmission design (RTD) of a multi-cell device-to-device (D2D) underlaid cellular network with imperfect channel state information (CSI). The bounded model is adopted to characterize the CSI impairment and the aim is to maximize the worst-case sum rate of the system. To protect cellular communications, it is assumed that the interference from all D2D transmitters to each base station (BS) is power-limited. It is first shown that the worst-case signal-to-interference-plus-noise ratio (SINR) of each D2D link can be obtained directly, while that of cellular links cannot be similarly found since the channel estimation error vectors of cellular links are coupled in the SINR expressions. To solve the nonconvex problem, the objective function of the original problem is replaced with its lower bound, and the resulted problem is decomposed into multiple semidefinite programming (SDP) subproblems which are convex and have computationally efficient solutions. An iterative RTD algorithm is then proposed to obtain a suboptimal solution. Simulation results show that D2D communication can significantly increase the performance of the conventional cellular systems while causing tolerable interference to cellular users. In addition, the proposed RTD algorithm outperforms the conventional non-robust transmission design greatly in terms of network spectral efficiency.
Porting nTorrent to ndnSIM
Venue: arXiv cs.NI 2018
Year: 2018

BitTorrent is a popular communication protocol for peer-to-peer file sharing. It uses a data-centric approach, wherein the data is decentralized and peers request each other for pieces of the file(s). Aspects of this process is similar to the Named Data Networking (NDN) architecture, but is realized completely at the application level on top of TCP/IP networking. nTorrent is a peer-to-peer file sharing application that is based on NDN. The goal of this project is to port the application onto ndnSIM to allow for simulation and testing.
ClassiNet -- Predicting Missing Features for Short-Text Classification
Venue: arXiv cs.CL 2018
Year: 2018

The fundamental problem in short-text classification is \emph{feature sparseness} -- the lack of feature overlap between a trained model and a test instance to be classified. We propose \emph{ClassiNet} -- a network of classifiers trained for predicting missing features in a given instance, to overcome the feature sparseness problem. Using a set of unlabeled training instances, we first learn binary classifiers as feature predictors for predicting whether a particular feature occurs in a given instance. Next, each feature predictor is represented as a vertex $v_i$ in the ClassiNet where a one-to-one correspondence exists between feature predictors and vertices. The weight of the directed edge $e_{ij}$ connecting a vertex $v_i$ to a vertex $v_j$ represents the conditional probability that given $v_i$ exists in an instance, $v_j$ also exists in the same instance. We show that ClassiNets generalize word co-occurrence graphs by considering implicit co-occurrences between features. We extract numerous features from the trained ClassiNet to overcome feature sparseness. In particular, for a given instance $\vec{x}$, we find similar features from ClassiNet that did not appear in $\vec{x}$, and append those features in the representation of $\vec{x}$. Moreover, we propose a method based on graph propagation to find features that are indirectly related to a given short-text. We evaluate ClassiNets on several benchmark datasets for short-text classification. Our experimental results show that by using ClassiNet, we can statistically significantly improve the accuracy in short-text classification tasks, without having to use any external resources such as thesauri for finding related features.
On the Selection of Anchors and Targets for Video Hyperlinking
Venue: arXiv cs.CV 2018
Year: 2018

A problem not well understood in video hyperlinking is what qualifies a fragment as an anchor or target. Ideally, anchors provide good starting points for navigation, and targets supplement anchors with additional details while not distracting users with irrelevant, false and redundant information. The problem is not trivial for intertwining relationship between data characteristics and user expectation. Imagine that in a large dataset, there are clusters of fragments spreading over the feature space. The nature of each cluster can be described by its size (implying popularity) and structure (implying complexity). A principle way of hyperlinking can be carried out by picking centers of clusters as anchors and from there reach out to targets within or outside of clusters with consideration of neighborhood complexity. The question is which fragments should be selected either as anchors or targets, in one way to reflect the rich content of a dataset, and meanwhile to minimize the risk of frustrating user experience. This paper provides some insights to this question from the perspective of hubness and local intrinsic dimensionality, which are two statistical properties in assessing the popularity and complexity of data space. Based these properties, two novel algorithms are proposed for low-risk automatic selection of anchors and targets.
The EcoLexicon Semantic Sketch Grammar: from Knowledge Patterns to Word Sketches
Venue: arXiv cs.CL 2018
Year: 2018

Many projects have applied knowledge patterns (KPs) to the retrieval of specialized information. Yet terminologists still rely on manual analysis of concordance lines to extract semantic information, since there are no user-friendly publicly available applications enabling them to find knowledge rich contexts (KRCs). To fill this void, we have created the KP-based EcoLexicon Semantic SketchGrammar (ESSG) in the well-known corpus query system Sketch Engine. For the first time, the ESSG is now publicly available inSketch Engine to query the EcoLexicon English Corpus. Additionally, reusing the ESSG in any English corpus uploaded by the user enables Sketch Engine to extract KRCs codifying generic-specific, part-whole, location, cause and function relations, because most of the KPs are domain-independent. The information is displayed in the form of summary lists (word sketches) containing the pairs of terms linked by a given semantic relation. This paper describes the process of building a KP-based sketch grammar with special focus on the last stage, namely, the evaluation with refinement purposes. We conducted an initial shallow precision and recall evaluation of the 64 English sketch grammar rules created so far for hyponymy, meronymy and causality. Precision was measured based on a random sample of concordances extracted from each word sketch type. Recall was assessed based on a random sample of concordances where known term pairs are found. The results are necessary for the improvement and refinement of the ESSG. The noise of false positives helped to further specify the rules, whereas the silence of false negatives allows us to find useful new patterns.
Data-efficient Neuroevolution with Kernel-Based Surrogate Models
Venue: arXiv cs.NE 2018
Year: 2018

Surrogate-assistance approaches have long been used in computationally expensive domains to improve the data-efficiency of optimization algorithms. Neuroevolution, however, has so far resisted the application of these techniques because it requires the surrogate model to make fitness predictions based on variable topologies, instead of a vector of parameters. Our main insight is that we can sidestep this problem by using kernel-based surrogate models, which require only the definition of a distance measure between individuals. Our second insight is that the well-established Neuroevolution of Augmenting Topologies (NEAT) algorithm provides a computationally efficient distance measure between dissimilar networks in the form of "compatibility distance", initially designed to maintain topological diversity. Combining these two ideas, we introduce a surrogate-assisted neuroevolution algorithm that combines NEAT and a surrogate model built using a compatibility distance kernel. We demonstrate the data-efficiency of this new algorithm on the low dimensional cart-pole swing-up problem, as well as the higher dimensional half-cheetah running task. In both tasks the surrogate-assisted variant achieves the same or better results with several times fewer function evaluations as the original NEAT.