OpenRaiser/Intern-Atlas huggingface dataset:
Swarm Robots Inspired by Friendship Formation Process
Venue: arXiv cs.RO 2018
Year: 2018

Swarm robotic systems are systems in which multiple robots having simple functionality perform tasks through their cooperation, and are advantageous in that they can exhibit non-trivial macroscopic functions such as adaptability, fault tolerance, and scalability. We previously proposed a simple model of swarm formation inspired by friendship formation process in human society, and demonstrated via simulation that various non-trivial patterns emerge. In this study, we examine the applicability of the proposed model to a swarm robotic system. As a first step, we developed five robots and demonstrated via real-world experiments that the simulation results can be largely reproduced.
A Full End-to-End Semantic Role Labeler, Syntax-agnostic Over Syntax-aware?
Venue: arXiv cs.CL 2018
Year: 2018

Semantic role labeling (SRL) is to recognize the predicate-argument structure of a sentence, including subtasks of predicate disambiguation and argument labeling. Previous studies usually formulate the entire SRL problem into two or more subtasks. For the first time, this paper introduces an end-to-end neural model which unifiedly tackles the predicate disambiguation and the argument labeling in one shot. Using a biaffine scorer, our model directly predicts all semantic role labels for all given word pairs in the sentence without relying on any syntactic parse information. Specifically, we augment the BiLSTM encoder with a non-linear transformation to further distinguish the predicate and the argument in a given sentence, and model the semantic role labeling process as a word pair classification task by employing the biaffine attentional mechanism. Though the proposed model is syntax-agnostic with local decoder, it outperforms the state-of-the-art syntax-aware SRL systems on the CoNLL-2008, 2009 benchmarks for both English and Chinese. To our best knowledge, we report the first syntax-agnostic SRL model that surpasses all known syntax-aware models.
Secure Convolutional Neural Network using FHE
Venue: arXiv cs.CR 2018
Year: 2018

In this paper, a secure Convolutional Neural Network classifier is proposed using Fully Homomorphic Encryption (FHE). The secure classifier provides a user with the ability to out-source the computations to a powerful cloud server and/or setup a server to classify inputs without providing the model or revealing source data. To this end, a real number framework is developed over FHE by using a fixed point format with binary digits. This allows for real number computations for basic operators like addition, subtraction, and multiplication but also to include secure comparisons and max functions. Additionally, a rectified linear unit is designed and realized in the framework. Experimentally, the model was verified using a Convolutional Neural Network trained for handwritten digits. This encrypted implementation shows accurate results for all classification when compared against an unencrypted implementation.
Fake Sentence Detection as a Training Task for Sentence Encoding
Venue: arXiv cs.CL 2018
Year: 2018

Sentence encoders are typically trained on language modeling tasks with large unlabeled datasets. While these encoders achieve state-of-the-art results on many sentence-level tasks, they are difficult to train with long training cycles. We introduce fake sentence detection as a new training task for learning sentence encoders. We automatically generate fake sentences by corrupting original sentences from a source collection and train the encoders to produce representations that are effective at detecting fake sentences. This binary classification task turns to be quite efficient for training sentence encoders. We compare a basic BiLSTM encoder trained on this task with a strong sentence encoding models (Skipthought and FastSent) trained on a language modeling task. We find that the BiLSTM trains much faster on fake sentence detection (20 hours instead of weeks) using smaller amounts of data (1M instead of 64M sentences). Further analysis shows the learned representations capture many syntactic and semantic properties expected from good sentence representations.
Sample Mixed-Based Data Augmentation for Domestic Audio Tagging
Venue: arXiv cs.SD 2018
Year: 2018

Audio tagging has attracted increasing attention since last decade and has various potential applications in many fields. The objective of audio tagging is to predict the labels of an audio clip. Recently deep learning methods have been applied to audio tagging and have achieved state-of-the-art performance, which provides a poor generalization ability on new data. However due to the limited size of audio tagging data such as DCASE data, the trained models tend to result in overfitting of the network. Previous data augmentation methods such as pitch shifting, time stretching and adding background noise do not show much improvement in audio tagging. In this paper, we explore the sample mixed data augmentation for the domestic audio tagging task, including mixup, SamplePairing and extrapolation. We apply a convolutional recurrent neural network (CRNN) with attention module with log-scaled mel spectrum as a baseline system. In our experiments, we achieve an state-of-the-art of equal error rate (EER) of 0.10 on DCASE 2016 task4 dataset with mixup approach, outperforming the baseline system without data augmentation.
Sequence Labeling: A Practical Approach
Venue: arXiv cs.CL 2018
Year: 2018

We take a practical approach to solving sequence labeling problem assuming unavailability of domain expertise and scarcity of informational and computational resources. To this end, we utilize a universal end-to-end Bi-LSTM-based neural sequence labeling model applicable to a wide range of NLP tasks and languages. The model combines morphological, semantic, and structural cues extracted from data to arrive at informed predictions. The model's performance is evaluated on eight benchmark datasets (covering three tasks: POS-tagging, NER, and Chunking, and four languages: English, German, Dutch, and Spanish). We observe state-of-the-art results on four of them: CoNLL-2012 (English NER), CoNLL-2002 (Dutch NER), GermEval 2014 (German NER), Tiger Corpus (German POS-tagging), and competitive performance on the rest.
Large-Scale Learnable Graph Convolutional Networks
Venue: arXiv cs.LG 2018
Year: 2018

Convolutional neural networks (CNNs) have achieved great success on grid-like data such as images, but face tremendous challenges in learning from more generic data such as graphs. In CNNs, the trainable local filters enable the automatic extraction of high-level features. The computation with filters requires a fixed number of ordered units in the receptive fields. However, the number of neighboring units is neither fixed nor are they ordered in generic graphs, thereby hindering the applications of convolutional operations. Here, we address these challenges by proposing the learnable graph convolutional layer (LGCL). LGCL automatically selects a fixed number of neighboring nodes for each feature based on value ranking in order to transform graph data into grid-like structures in 1-D format, thereby enabling the use of regular convolutional operations on generic graphs. To enable model training on large-scale graphs, we propose a sub-graph training method to reduce the excessive memory and computational resource requirements suffered by prior methods on graph convolutions. Our experimental results on node classification tasks in both transductive and inductive learning settings demonstrate that our methods can achieve consistently better performance on the Cora, Citeseer, Pubmed citation network, and protein-protein interaction network datasets. Our results also indicate that the proposed methods using sub-graph training strategy are more efficient as compared to prior approaches.
Augmenting word2vec with latent Dirichlet allocation within a clinical application
Venue: arXiv cs.CL 2018
Year: 2018

This paper presents three hybrid models that directly combine latent Dirichlet allocation and word embedding for distinguishing between speakers with and without Alzheimer's disease from transcripts of picture descriptions. Two of our models get F-scores over the current state-of-the-art using automatic methods on the DementiaBank dataset.
Text Classification using Capsules
Venue: arXiv cs.CL 2018
Year: 2018

This paper presents an empirical exploration of the use of capsule networks for text classification. While it has been shown that capsule networks are effective for image classification, their validity in the domain of text has not been explored. In this paper, we show that capsule networks indeed have the potential for text classification and that they have several advantages over convolutional neural networks. We further suggest a simple routing method that effectively reduces the computational complexity of dynamic routing. We utilized seven benchmark datasets to demonstrate that capsule networks, along with the proposed routing method provide comparable results.
Enhancing camera surveillance using computer vision: a research note
Venue: arXiv cs.CY 2018
Year: 2018

$\mathbf{Purpose}$ - The growth of police operated surveillance cameras has out-paced the ability of humans to monitor them effectively. Computer vision is a possible solution. An ongoing research project on the application of computer vision within a municipal police department is described. The paper aims to discuss these issues. $\mathbf{Design/methodology/approach}$ - Following the demystification of computer vision technology, its potential for police agencies is developed within a focus on computer vision as a solution for two common surveillance camera tasks (live monitoring of multiple surveillance cameras and summarizing archived video files). Three unaddressed research questions (can specialized computer vision applications for law enforcement be developed at this time, how will computer vision be utilized within existing public safety camera monitoring rooms, and what are the system-wide impacts of a computer vision capability on local criminal justice systems) are considered. $\mathbf{Findings}$ - Despite computer vision becoming accessible to law enforcement agencies the impact of computer vision has not been discussed or adequately researched. There is little knowledge of computer vision or its potential in the field. $\mathbf{Originality/value}$ - This paper introduces and discusses computer vision from a law enforcement perspective and will be valuable to police personnel tasked with monitoring large camera networks and considering computer vision as a system upgrade.
An Entropy Analysis based Intrusion Detection System for Controller Area Network in Vehicles
Venue: arXiv cs.CR 2018
Year: 2018

Dozens of Electronic Control Units (ECUs) can be found on modern vehicles for safety and driving assistance. These ECUs also introduce new security vulnerabilities as recent attacks have been reported by plugging the in-vehicle system or through wireless access. In this paper, we focus on the security of the Controller Area Network (CAN), which is a standard for communication among ECUs. CAN bus by design does not have sufficient security features to protect it from insider or outsider attacks. Intrusion detection system (IDS) is one of the most effective ways to enhance vehicle security on the insecure CAN bus protocol. We propose a new IDS based on the entropy of the identifier bits in CAN messages. The key observation is that all the known CAN message injection attacks need to alter the CAN ID bits and analyzing the entropy of such bits can be an effective way to detect those attacks. We collected real CAN messages from a vehicle (2016 Ford Fusion) and performed simulated message injection attacks. The experimental results showed that our entropy based IDS can successfully detect all the injection attacks without disrupting the communication on CAN.
Time Perception Machine: Temporal Point Processes for the When, Where and What of Activity Prediction
Venue: arXiv cs.CV 2018
Year: 2018

Numerous powerful point process models have been developed to understand temporal patterns in sequential data from fields such as health-care, electronic commerce, social networks, and natural disaster forecasting. In this paper, we develop novel models for learning the temporal distribution of human activities in streaming data (e.g., videos and person trajectories). We propose an integrated framework of neural networks and temporal point processes for predicting when the next activity will happen. Because point processes are limited to taking event frames as input, we propose a simple yet effective mechanism to extract features at frames of interest while also preserving the rich information in the remaining frames. We evaluate our model on two challenging datasets. The results show that our model outperforms traditional statistical point process approaches significantly, demonstrating its effectiveness in capturing the underlying temporal dynamics as well as the correlation within sequential activities. Furthermore, we also extend our model to a joint estimation framework for predicting the timing, spatial location, and category of the activity simultaneously, to answer the when, where, and what of activity prediction.
A Transfer Learning based Feature-Weak-Relevant Method for Image Clustering
Venue: arXiv cs.CV 2018
Year: 2018

Image clustering is to group a set of images into disjoint clusters in a way that images in the same cluster are more similar to each other than to those in other clusters, which is an unsupervised or semi-supervised learning process. It is a crucial and challenging task in machine learning and computer vision. The performances of existing image clustering methods have close relations with features used for clustering, even if unsupervised coding based methods have improved the performances a lot. To reduce the effect of clustering features, we propose a feature-weak-relevant method for image clustering. The proposed method converts an unsupervised clustering process into an alternative iterative process of unsupervised learning and transfer learning. The clustering process firstly starts up from handcrafted features based image clustering to estimate an initial label for every image, and secondly use a proposed sampling strategy to choose images with reliable labels to feed a transfer-learning model to learn representative features that can be used for next round of unsupervised learning. In this manner, image clustering is iteratively optimized. What's more, the handcrafted features are used to boot up the clustering process, and just have a little effect on the final performance; therefore, the proposed method is feature-weak-relevant. Experimental results on six kinds of public available datasets show that the proposed method outperforms state of the art methods and depends less on the employed features at the same time.
Towards Audio to Scene Image Synthesis using Generative Adversarial Network
Venue: arXiv cs.CL 2018
Year: 2018

Humans can imagine a scene from a sound. We want machines to do so by using conditional generative adversarial networks (GANs). By applying the techniques including spectral norm, projection discriminator and auxiliary classifier, compared with naive conditional GAN, the model can generate images with better quality in terms of both subjective and objective evaluations. Almost three-fourth of people agree that our model have the ability to generate images related to sounds. By inputting different volumes of the same sound, our model output different scales of changes based on the volumes, showing that our model truly knows the relationship between sounds and images to some extent.
Proving Program Properties as First-Order Satisfiability
Venue: arXiv cs.LO 2018
Year: 2018

Program semantics can often be expressed as a (many-sorted) first-order theory S, and program properties as sentences $φ$ which are intended to hold in the canonical model of such a theory, which is often incomputable. Recently, we have shown that properties $φ$ expressed as the existential closure of a boolean combination of atoms can be disproved by just finding a model of S and the negation $\negφ$ of $φ$. Furthermore, this idea works quite well in practice due to the existence of powerful tools for the automatic generation of models for (many-sorted) first-order theories. In this paper we extend our previous result to arbitrary properties, expressed as sentences without any special restriction. Consequently, one can prove a program property $φ$ by just finding a model of an appropriate theory (including S and possibly something else) and an appropriate first-order formula related to $φ$. Beyond its possible theoretical interest, we show that our results can also be of practical use in several respects.
Image Inpainting Based on a Novel Criminisi Algorithm
Venue: arXiv cs.GR 2018
Year: 2018

In view of the problem of image inpainting error continuation and the deviation of finding best match block, an improved Criminisi algorithm is proposed. The improvement was mainly embodied in two aspects. In the repairing order aspect, we redefine the calculation formula of the priority. In order to solve the problem of error continuation caused by local confidence item updating, the mean value of Manhattan distance is used for replace the confidence item. In the matching strategy aspect, finding the best match block not only depend on the difference of the two pixels, but also consider the matching region. Therefore, Euclidean distance is introduced. Experiments confirm that the improved algorithm can overcome the insufficiencies of the original algorithm. The repairing effect has been improved, and the results have a better visual appearance.
A New Hybrid Consensus Protocol: Deterministic Proof Of Work
Venue: arXiv cs.CR 2018
Year: 2018

The Decentralized-Consistent-Scale (DCS) Triangle defines three dimensions that illustrate the tradeoffs of the blockchain consensus mechanism. In this paper, we propose a new hybrid consensus protocol, called Deterministic Proof of Work (DPoW), which can reach high levels of scalability and consistency without significant reduction to decentralization. Our protocol introduces a Map-reduce PoW mining algorithm to perform alongside Practical Byzantine Fault Tolerance (PBFT) verification, which together allow for transactions to be confirmed immediately, largely improving scalability. In addition, the protocol is designed such that forking cannot occur, ensuring strong consistency and security against a multitude of attacks. The Map-reduce PoW mining process ensures that no single entity can control the network, guaranteeing decentralization. We analyzed the security of our protocol by evaluating the possibility of double spending attacks, and furthermore, conducted experiments which demonstrate our claims.
Effective Unsupervised Author Disambiguation with Relative Frequencies
Venue: arXiv cs.IR 2018
Year: 2018

This work addresses the problem of author name homonymy in the Web of Science. Aiming for an efficient, simple and straightforward solution, we introduce a novel probabilistic similarity measure for author name disambiguation based on feature overlap. Using the researcher-ID available for a subset of the Web of Science, we evaluate the application of this measure in the context of agglomeratively clustering author mentions. We focus on a concise evaluation that shows clearly for which problem setups and at which time during the clustering process our approach works best. In contrast to most other works in this field, we are sceptical towards the performance of author name disambiguation methods in general and compare our approach to the trivial single-cluster baseline. Our results are presented separately for each correct clustering size as we can explain that, when treating all cases together, the trivial baseline and more sophisticated approaches are hardly distinguishable in terms of evaluation results. Our model shows state-of-the-art performance for all correct clustering sizes without any discriminative training and with tuning only one convergence parameter.
Addressing Client Needs for Cloud Computing using Formal Foundations
Venue: arXiv cs.SE 2018
Year: 2018

Cloud-enabled large-scale distributed systems orchestrate resources and services from various providers in order to deliver high-quality software solutions to the end users. The space and structure created by such technological advancements are immense sources of information and impose a high complexity and heterogeneity, which might lead to unexpected failures. In this chapter, we present a model that coordinates the multi-cloud interaction through the specification, validation, and verification of a middle-ware exploiting monitoring and adaptation processes. The monitoring processes handle collecting meaningful data and assessing the state of components, while the adaptation processes restore the system as dictated by the evolution needs and sudden changes in the operating environment conditions. We employ Abstract State Machines to specify the models and we further make use of the ASMETA framework to simulate and validate them. Desired properties of the system are defined and analysed with the aid of the Computation Tree Logic.
Relational dynamic memory networks
Venue: arXiv cs.AI 2018
Year: 2018

Neural networks excel in detecting regular patterns but are less successful in representing and manipulating complex data structures, possibly due to the lack of an external memory. This has led to the recent development of a new line of architectures known as Memory-Augmented Neural Networks (MANNs), each of which consists of a neural network that interacts with an external memory matrix. However, this RAM-like memory matrix is unstructured and thus does not naturally encode structured objects. Here we design a new MANN dubbed Relational Dynamic Memory Network (RMDN) to bridge the gap. Like existing MANNs, RMDN has a neural controller but its memory is structured as multi-relational graphs. RMDN uses the memory to represent and manipulate graph-structured data in response to query; and as a neural network, RMDN is trainable from labeled data. Thus RMDN learns to answer queries about a set of graph-structured objects without explicit programming. We evaluate the capability of RMDN on several important prediction problems, including software vulnerability, molecular bioactivity and chemical-chemical interaction. Results demonstrate the efficacy of the proposed model.
Simple Root Cause Analysis by Separable Likelihoods
Venue: arXiv cs.LG 2018
Year: 2018

Root Cause Analysis for Anomalies is challenging because of the trade-off between the accuracy and its explanatory friendliness, required for industrial applications. In this paper we propose a framework for simple and friendly RCA within the Bayesian regime under certain restrictions (that Hessian at the mode is diagonal, here referred to as \emph{separability}) imposed on the predictive posterior. We show that this assumption is satisfied for important base models, including Multinomal, Dirichlet-Multinomial and Naive Bayes. To demonstrate the usefulness of the framework, we embed it into the Bayesian Net and validate on web server error logs (real world data set).