OpenRaiser/Intern-Atlas huggingface dataset:
Leveraging Coding Techniques for Speeding up Distributed Computing
Venue: arXiv cs.IT 2018
Year: 2018

Large scale clusters leveraging distributed computing frameworks such as MapReduce routinely process data that are on the orders of petabytes or more. The sheer size of the data precludes the processing of the data on a single computer. The philosophy in these methods is to partition the overall job into smaller tasks that are executed on different servers; this is called the map phase. This is followed by a data shuffling phase where appropriate data is exchanged between the servers. The final so-called reduce phase, completes the computation. One potential approach, explored in prior work for reducing the overall execution time is to operate on a natural tradeoff between computation and communication. Specifically, the idea is to run redundant copies of map tasks that are placed on judiciously chosen servers. The shuffle phase exploits the location of the nodes and utilizes coded transmission. The main drawback of this approach is that it requires the original job to be split into a number of map tasks that grows exponentially in the system parameters. This is problematic, as we demonstrate that splitting jobs too finely can in fact adversely affect the overall execution time. In this work we show that one can simultaneously obtain low communication loads while ensuring that jobs do not need to be split too finely. Our approach uncovers a deep relationship between this problem and a class of combinatorial structures called resolvable designs. Appropriate interpretation of resolvable designs can allow for the development of coded distributed computing schemes where the splitting levels are exponentially lower than prior work. We present experimental results obtained on Amazon EC2 clusters for a widely known distributed algorithm, namely TeraSort. We obtain over 4.69$\times$ improvement in speedup over the baseline approach and more than 2.6$\times$ over current state of the art.
Mining Open Government Data Used in Scientific Research
Venue: arXiv cs.CY 2018
Year: 2018

In the following paper, we describe results from mining citations, mentions, and links to open government data (OGD) in peer-reviewed literature. We inductively develop a method for categorizing how OGD are used by different research communities, and provide descriptive statistics about the publication years, publication outlets, and OGD sources. Our results demonstrate that, 1. The use of OGD in research is steadily increasing from 2009 to 2016; 2. Researchers use OGD from 96 different open government data portals, with data.gov.uk and data.gov being the most frequent sources; and, 3.Contrary to previous findings, we provide evidence suggesting that OGD from developing nations, notably India and Kenya, are being frequently used to fuel scientific discoveries. The findings of this paper contribute to ongoing research agendas aimed at tracking the impact of open government data initiatives, and provides an initial description of how open government data are valuable to diverse scientific research communities.
Learning to Match
Venue: arXiv cs.IR 2018
Year: 2018

Booking.com is a virtual two-sided marketplace where guests and accommodation providers are the two distinct stakeholders. They meet to satisfy their respective and different goals. Guests want to be able to choose accommodations from a huge and diverse inventory, fast and reliably within their requirements and constraints. Accommodation providers desire to reach a reliable and large market that maximizes their revenue. Finding the best accommodation for the guests, a problem typically addressed by the recommender systems community, and finding the best audience for the accommodation providers, are key pieces of a good platform. This work describes how Booking.com extends such approach, enabling the guests themselves to find the best accommodation by helping them to discover their needs and restrictions, what the market can actually offer, reinforcing good decisions, discouraging bad ones, etc. turning the platform into a decision process advisor, as opposed to a decision maker. Booking.com implements this idea with hundreds of Machine Learned Models, all of them validated through rigorous Randomized Controlled Experiments. We further elaborate on model types, techniques, methodological issues and challenges that we have faced.
URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection
Venue: arXiv cs.CR 2018
Year: 2018

Malicious URLs host unsolicited content and are used to perpetrate cybercrimes. It is imperative to detect them in a timely manner. Traditionally, this is done through the usage of blacklists, which cannot be exhaustive, and cannot detect newly generated malicious URLs. To address this, recent years have witnessed several efforts to perform Malicious URL Detection using Machine Learning. The most popular and scalable approaches use lexical properties of the URL string by extracting Bag-of-words like features, followed by applying machine learning models such as SVMs. There are also other features designed by experts to improve the prediction performance of the model. These approaches suffer from several limitations: (i) Inability to effectively capture semantic meaning and sequential patterns in URL strings; (ii) Requiring substantial manual feature engineering; and (iii) Inability to handle unseen features and generalize to test data. To address these challenges, we propose URLNet, an end-to-end deep learning framework to learn a nonlinear URL embedding for Malicious URL Detection directly from the URL. Specifically, we apply Convolutional Neural Networks to both characters and words of the URL String to learn the URL embedding in a jointly optimized framework. This approach allows the model to capture several types of semantic information, which was not possible by the existing models. We also propose advanced word-embeddings to solve the problem of too many rare words observed in this task. We conduct extensive experiments on a large-scale dataset and show a significant performance gain over existing methods. We also conduct ablation studies to evaluate the performance of various components of URLNet.
Tracking all members of a honey bee colony over their lifetime
Venue: arXiv cs.CV 2018
Year: 2018

Computational approaches to the analysis of collective behavior in social insects increasingly rely on motion paths as an intermediate data layer from which one can infer individual behaviors or social interactions. Honey bees are a popular model for learning and memory. Previous experience has been shown to affect and modulate future social interactions. So far, no lifetime history observations have been reported for all bees of a colony. In a previous work we introduced a tracking system customized to track up to $4000$ bees over several weeks. In this contribution we present an in-depth description of the underlying multi-step algorithm which both produces the motion paths, and also improves the marker decoding accuracy significantly. We automatically tracked ${\sim}2000$ marked honey bees over 10 weeks with inexpensive recording hardware using markers without any error correction bits. We found that the proposed two-step tracking reduced incorrect ID decodings from initially ${\sim}13\%$ to around $2\%$ post-tracking. Alongside this paper, we publish the first trajectory dataset for all bees in a colony, extracted from ${\sim} 4$ million images. We invite researchers to join the collective scientific effort to investigate this intriguing animal system. All components of our system are open-source.
Research and Implementation of Global Path Planning for Unmanned Surface Vehicle Based on Electronic Chart
Venue: arXiv cs.RO 2018
Year: 2018

Unmanned Surface Vehicle (USV) is a new type of intelligent surface craft, and global path planning is the key technology of USV research, which can reflect the intelligent level of USV. In order to solve the problem of global path planning of USV, this paper proposes an improved A* algorithm for sailing cost optimization based on electronic charts. This paper uses the S-57 electronic chart to realize the establishment of the octree grid environment model, and proposes an improved A* algorithm based on sailing safety weight, pilot quantity and path curve smoothing to ensure the safety of the route, reduce the planning time, and improve path smoothness. The simulation results show that the environmental model construction method and the improved A* algorithm can generate safe and reasonable global path.
Video Event Recognition and Anomaly Detection by Combining Gaussian Process and Hierarchical Dirichlet Process Models
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we present an unsupervised learning framework for analyzing activities and interactions in surveillance videos. In our framework, three levels of video events are connected by Hierarchical Dirichlet Process (HDP) model: low-level visual features, simple atomic activities, and multi-agent interactions. Atomic activities are represented as distribution of low-level features, while complicated interactions are represented as distribution of atomic activities. This learning process is unsupervised. Given a training video sequence, low-level visual features are extracted based on optic flow and then clustered into different atomic activities and video clips are clustered into different interactions. The HDP model automatically decide the number of clusters, i.e. the categories of atomic activities and interactions. Based on the learned atomic activities and interactions, a training dataset is generated to train the Gaussian Process (GP) classifier. Then the trained GP models work in newly captured video to classify interactions and detect abnormal events in real time. Furthermore, the temporal dependencies between video events learned by HDP-Hidden Markov Models (HMM) are effectively integrated into GP classifier to enhance the accuracy of the classification in newly captured videos. Our framework couples the benefits of the generative model (HDP) with the discriminant model (GP). We provide detailed experiments showing that our framework enjoys favorable performance in video event classification in real-time in a crowded traffic scene.
Shapes Characterization on Address Event Representation Using Histograms of Oriented Events and an Extended LBP Approach
Venue: arXiv cs.CV 2018
Year: 2018

Address Event Representation is a thriving technology that could change digital image processing paradigm. This paper proposes a methodology to characterize the shape of objects using the streaming of asynchronous events. A new descriptor that enhances spikes connectivity is associated with two oriented histogram based representations. This paper uses these features to develop both a non-supervised and a supervised multi-classification framework to recognize poker symbols from the Poker-DVS public dataset. The aforementioned framework, which uses a very limited number of events and a simple class modeling, yields results that challenge more sophisticated methodologies proposed by the state of the art. A feature family based on context shapes is applied to the more challenging 2015 Poker-DVS dataset with a supervised classifier obtaining an accuracy of 98.5 %. The system is also applied to the MNIST-DVS dataset yielding an accuracy of 94.6 % and 96.3 % on digit recognition, for scales 4 and 8 respectively.
InferBeam: A Fast Beam Alignment Protocol for Millimeter-wave Networking
Venue: arXiv cs.NI 2018
Year: 2018

We introduce fast millimeter-wave base station (BS) and its antenna sector selection for user equipment based on its location. Using a conditional random field inference model with specially designed parameters, which are robust to change of environment, InferBeam allows the use of measurement samples on best beam selection at a small number of locations to infer the rest dynamically. Compared to beam-sweeping based approaches in the literature, InferBeam can drastically reduce the setup cost for beam alignment for a new environment, and also the latency in acquiring a new beam under intermittent blockage. We have evaluated InferBeam using a discrete event simulation. Our results indicate that the system can make best beam selection for 98% of locations in test environments comprising smallsized apartment or office spaces, while sampling fewer than 1% of locations. InferBeam is a complete protocol for best beam inference that can be integrated into millimeter-wave standards for accelerating the much-needed fast and economic beam alignment capability.
A Study of WhatsApp Usage Patterns and Prediction Models without Message Content
Venue: arXiv cs.CY 2018
Year: 2018

Internet social networks have become a ubiquitous application allowing people to easily share text, pictures, and audio and video files. Popular networks include WhatsApp, Facebook, Reddit and LinkedIn. We present an extensive study of the usage of the WhatsApp social network, an Internet messaging application that is quickly replacing SMS messaging. In order to better understand people's use of the network, we provide an analysis of over 6 million messages from over 100 users, with the objective of building demographic prediction models using activity data. We performed extensive statistical and numerical analysis of the data and found significant differences in WhatsApp usage across people of different genders and ages. We also inputted the data into the Weka data mining package and studied models created from decision tree and Bayesian network algorithms. We found that different genders and age demographics had significantly different usage habits in almost all message and group attributes. We also noted differences in users' group behavior and created prediction models, including the likelihood a given group would have relatively more file attachments, if a group would contain a larger number of participants, a higher frequency of activity, quicker response times and shorter messages. We were successful in quantifying and predicting a user's gender and age demographic. Similarly, we were able to predict different types of group usage. All models were built without analyzing message content. We present a detailed discussion about the specific attributes that were contained in all predictive models and suggest possible applications based on these results.
State of the Practice for GIS Software
Venue: arXiv cs.SE 2018
Year: 2018

We present a reproducible method to analyze the state of software development practices in a given scientific domain and apply this method to Geographic Information Systems (GIS). The analysis is based on grading a set of 30 GIS products using a template of 56 questions based on 13 software qualities. The products range in scope and purpose from a complete desktop GIS systems, to stand-alone tools, to programming libraries/packages. The final ranking of the products is determined using the Analytic Hierarchy Process (AHP), a multicriteria decision making method that focuses on relative comparisons between products, rather than directly measuring qualities. The results reveal concerns regarding the correctness, maintainability, transparency and reproducibility of some GIS software. Three recommendations are presented as feedback to the GIS community: 1) Ensure each project has a requirements specification document, 2) Provide a wealth of support methods, such as an IRC (Internet Relay Chat) channel, a Stack Exchange tag for new questions, or opening the issue tracker for support requests, as well as the more traditional email-based methods, and, 3) Design product websites for maximum transparency (of the development process), for open source projects, provide a developer's guide.
Hybrid Controller for Wind Turbine Generators to Ensure Adequate Frequency Response in Power Networks
Venue: arXiv cs.SY 2018
Year: 2018

Converter-interfaced power sources (CIPS) are hybrid control systems as they may switch between multiple operating modes. Due to increasing penetration, the hybrid behavior of CIPS, such as, wind turbine generators (WTG), may have significant impact on power system dynamics. In this paper, the frequency dynamics under inertia emulation and primary support from WTG is studied. A mode switching for WTG to ensure adequate frequency response is proposed. The switching instants are determined by our proposed concept of a region of safety (ROS), which is the initial set of safe trajectories. The barrier certificate methodology is employed to derive a new algorithm to obtain and enlarge the ROS for the given desired safe limits and the worst-case disturbance scenarios. Then critical switching instants and a safe recovery procedure are found. In addition, the emulated inertia and load-damping effect is derived in the time frame of inertia and primary frequency response, respectively. The theoretical results under critical cases are consistent with simulations and can be used as guidance for practical control design.
Minimizing Latency to Support VR Social Interactions over Wireless Cellular Systems via Bandwidth Allocation
Venue: arXiv cs.IT 2018
Year: 2018

Immersive social interactions of mobile users are soon to be enabled within a virtual space, by means of virtual reality (VR) technologies and wireless cellular systems. In a VR mobile social network, the states of all interacting users should be updated synchronously and with low latency via two-way communications with edge computing servers. The resulting end-to-end latency depends on the relationship between the virtual and physical locations of the wireless VR users and of the edge servers. In this work, the problem of analyzing and optimizing the end-to-end latency is investigated for a simple network topology, yielding important insights into the interplay between physical and virtual geometries.
GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders
Venue: arXiv cs.LG 2018
Year: 2018

Deep learning on graphs has become a popular research topic with many applications. However, past work has concentrated on learning graph embedding tasks, which is in contrast with advances in generative models for images and text. Is it possible to transfer this progress to the domain of graphs? We propose to sidestep hurdles associated with linearization of such discrete structures by having a decoder output a probabilistic fully-connected graph of a predefined maximum size directly at once. Our method is formulated as a variational autoencoder. We evaluate on the challenging task of molecule generation.
Local Contrast Learning
Venue: arXiv cs.LG 2018
Year: 2018

Learning a deep model from small data is yet an opening and challenging problem. We focus on one-shot classification by deep learning approach based on a small quantity of training samples. We proposed a novel deep learning approach named Local Contrast Learning (LCL) based on the key insight about a human cognitive behavior that human recognizes the objects in a specific context by contrasting the objects in the context or in her/his memory. LCL is used to train a deep model that can contrast the recognizing sample with a couple of contrastive samples randomly drawn and shuffled. On one-shot classification task on Omniglot, the deep model based LCL with 122 layers and 1.94 millions of parameters, which was trained on a tiny dataset with only 60 classes and 20 samples per class, achieved the accuracy 97.99% that outperforms human and state-of-the-art established by Bayesian Program Learning (BPL) trained on 964 classes. LCL is a fundamental idea which can be applied to alleviate parametric model's overfitting resulted by lack of training samples.
Tubule segmentation of fluorescence microscopy images based on convolutional neural networks with inhomogeneity correction
Venue: arXiv cs.CV 2018
Year: 2018

Fluorescence microscopy has become a widely used tool for studying various biological structures of in vivo tissue or cells. However, quantitative analysis of these biological structures remains a challenge due to their complexity which is exacerbated by distortions caused by lens aberrations and light scattering. Moreover, manual quantification of such image volumes is an intractable and error-prone process, making the need for automated image analysis methods crucial. This paper describes a segmentation method for tubular structures in fluorescence microscopy images using convolutional neural networks with data augmentation and inhomogeneity correction. The segmentation results of the proposed method are visually and numerically compared with other microscopy segmentation methods. Experimental results indicate that the proposed method has better performance with correctly segmenting and identifying multiple tubular structures compared to other methods.
To the problem of "The Instrumental complex for ontological engineering purpose" software system design
Venue: arXiv cs.AI 2018
Year: 2018

The given work describes methodological principles of design instrumental complex of ontological purpose. Instrumental complex intends for the implementation of the integrated information technologies automated build of domain ontologies. Results focus on enhancing the effectiveness of the automatic analysis and understanding of natural-language texts, building of knowledge description of subject areas (primarily in the area of science and technology) and for interdisciplinary research in conjunction with the solution of complex problems.
Towards a Lightweight Multi-Cloud DSL for Elastic and Transferable Cloud-native Applications
Venue: arXiv cs.SE 2018
Year: 2018

Cloud-native applications are intentionally designed for the cloud in order to leverage cloud platform features like horizontal scaling and elasticity - benefits coming along with cloud platforms. In addition to classical (and very often static) multi-tier deployment scenarios, cloud-native applications are typically operated on much more complex but elastic infrastructures. Furthermore, there is a trend to use elastic container platforms like Kubernetes, Docker Swarm or Apache Mesos. However, especially multi-cloud use cases are astonishingly complex to handle. In consequence, cloud-native applications are prone to vendor lock-in. Very often TOSCA-based approaches are used to tackle this aspect. But, these application topology defining approaches are limited in supporting multi-cloud adaption of a cloud-native application at runtime. In this paper, we analyzed several approaches to define cloud-native applications being multi-cloud transferable at runtime. We have not found an approach that fully satisfies all of our requirements. Therefore we introduce a solution proposal that separates elastic platform definition from cloud application definition. We present first considerations for a domain specific language for application definition and demonstrate evaluation results on the platform level showing that a cloud-native application can be transferred between different cloud service providers like Azure and Google within minutes and without downtime. The evaluation covers public and private cloud service infrastructures provided by Amazon Web Services, Microsoft Azure, Google Compute Engine and OpenStack.