OpenRaiser/Intern-Atlas huggingface dataset:
Learning Cross-Domain Representation with Multi-Graph Neural Network
Venue: arXiv cs.LG 2019
Year: 2019

Learning effective embedding has been proved to be useful in many real-world problems, such as recommender systems, search ranking and online advertisement. However, one of the challenges is data sparsity in learning large-scale item embedding, as users' historical behavior data are usually lacking or insufficient in an individual domain. In fact, user's behaviors from different domains regarding the same items are usually relevant. Therefore, we can learn complete user behaviors to alleviate the sparsity using complementary information from correlated domains. It is intuitive to model users' behaviors using graph, and graph neural networks (GNNs) have recently shown the great power for representation learning, which can be used to learn item embedding. However, it is challenging to transfer the information across domains and learn cross-domain representation using the existing GNNs. To address these challenges, in this paper, we propose a novel model - Deep Multi-Graph Embedding (DMGE) to learn cross-domain representation. Specifically, we first construct a multi-graph based on users' behaviors from different domains, and then propose a multi-graph neural network to learn cross-domain representation in an unsupervised manner. Particularly, we present a multiple-gradient descent optimizer for efficiently training the model. We evaluate our approach on various large-scale real-world datasets, and the experimental results show that DMGE outperforms other state-of-art embedding methods in various tasks.
Visual Model-predictive Localization for Computationally Efficient Autonomous Racing of a 72-gram Drone
Venue: arXiv cs.RO 2019
Year: 2019

Drone racing is becoming a popular e-sport all over the world, and beating the best human drone race pilots has quickly become a new major challenge for artificial intelligence and robotics. In this paper, we propose a strategy for autonomous drone racing which is computationally more efficient than navigation methods like visual inertial odometry and simultaneous localization and mapping. This fast light-weight vision-based navigation algorithm estimates the position of the drone by fusing race gate detections with model dynamics predictions. Theoretical analysis and simulation results show the clear advantage compared to Kalman filtering when dealing with the relatively low frequency visual updates and occasional large outliers that occur in fast drone racing. Flight tests are performed on a tiny racing quadrotor named "Trashcan", which was equipped with a Jevois smart-camera for a total of 72g. The test track consists of 3 laps around a 4-gate racing track. The gates spaced 4 meters apart and can be displaced from their supposed position. An average speed of 2m/s is achieved while the maximum speed is 2.6m/s. To the best of our knowledge, this flying platform is the smallest autonomous racing drone in the world and is 6 times lighter than the existing lightest autonomous racing drone setup (420g), while still being one of the fastest autonomous racing drones in the world.
Perceptions of Gender Diversity's impact on mood in software development teams
Venue: arXiv cs.SE 2019
Year: 2019

Recent studies show that gender diversity in IT teams has a positive impact on the software development process. However, there is still a great gender inequality. The aim of our study was to examine how the working atmosphere depends on the gender differentiation of IT teams. The analysis of the results of the interviews and questionnaires showed that the atmosphere in gender-differentiated teams is more pleasant compared to purely male ones. The paper also discusses the problem of gender discrimination, which, according to the results of the study, unfortunately still exists and affects the working atmosphere. Finally, we looked at ways to reduce the gender inequity, where it turned out that soft approaches such as dedicated training, workshops to show the human face of the IT industry are preferred.
A Behavior-Based Ontology for Supporting Automated Assessment of Interactive Systems
Venue: arXiv cs.HC 2019
Year: 2019

Nowadays many software development frameworks implement Behavior-Driven Development (BDD) as a mean of automating the test of interactive systems under construction. Automated testing helps to simulate user's action on the User Interface and therefore check if the system behaves properly and in accordance to Scenarios that describe functional requirements. However, most of tools supporting BDD requires that tests should be written using low-level events and components that only exist when the system is already implemented. As a consequence of such low-level of abstraction, BDD tests can hardly be reused with diverse artifacts and with versions of the system. To address this problem, this paper proposes to raise the abstraction level by the means of a behavior-based ontology that is aimed at supporting test automation. The paper presents an ontology and an on-tology-based approach for automating the test of functional requirements of interactive systems. With the help of a case study for the flight tickets e-commerce domain, we demonstrate how tests written using our ontology can be used to assess functional requirements using different artifacts, from low-fidelity to full-fledged UI Prototypes.
From Here to There: Video Inbetweening Using Direct 3D Convolutions
Venue: arXiv cs.CV 2019
Year: 2019

We consider the problem of generating plausible and diverse video sequences, when we are only given a start and an end frame. This task is also known as inbetweening, and it belongs to the broader area of stochastic video generation, which is generally approached by means of recurrent neural networks (RNN). In this paper, we propose instead a fully convolutional model to generate video sequences directly in the pixel domain. We first obtain a latent video representation using a stochastic fusion mechanism that learns how to incorporate information from the start and end frames. Our model learns to produce such latent representation by progressively increasing the temporal resolution, and then decode in the spatiotemporal domain using 3D convolutions. The model is trained end-to-end by minimizing an adversarial loss. Experiments on several widely-used benchmark datasets show that it is able to generate meaningful and diverse in-between video sequences, according to both quantitative and qualitative evaluations.
Hardness of Distributed Optimization
Venue: arXiv cs.DS 2019
Year: 2019

This paper studies lower bounds for fundamental optimization problems in the CONGEST model. We show that solving problems exactly in this model can be a hard task, by providing $\tildeΩ(n^2)$ lower bounds for cornerstone problems, such as minimum dominating set (MDS), Hamiltonian path, Steiner tree and max-cut. These are almost tight, since all of these problems can be solved optimally in $O(n^2)$ rounds. Moreover, we show that even in bounded-degree graphs and even in simple graphs with maximum degree 5 and logarithmic diameter, it holds that various tasks, such as finding a maximum independent set (MaxIS) or a minimum vertex cover, are still difficult, requiring a near-tight number of $\tildeΩ(n)$ rounds. Furthermore, we show that in some cases even approximations are difficult, by providing an $\tildeΩ(n^2)$ lower bound for a $(7/8+ε)$-approximation for MaxIS, and a nearly-linear lower bound for an $O(\log{n})$-approximation for the $k$-MDS problem for any constant $k \geq 2$, as well as for several variants of the Steiner tree problem. Our lower bounds are based on a rich variety of constructions that leverage novel observations, and reductions among problems that are specialized for the CONGEST model. However, for several additional approximation problems, as well as for exact computation of some central problems in $P$, such as maximum matching and max flow, we show that such constructions cannot be designed, by which we exemplify some limitations of this framework.
Devil in the Detail: Attack Scenarios in Industrial Applications
Venue: arXiv cs.CR 2019
Year: 2019

In the past years, industrial networks have become increasingly interconnected and opened to private or public networks. This leads to an increase in efficiency and manageability, but also increases the attack surface. Industrial networks often consist of legacy systems that have not been designed with security in mind. In the last decade, an increase in attacks on cyber-physical systems was observed, with drastic consequences on the physical work. In this work, attack vectors on industrial networks are categorised. A real-world process is simulated, attacks are then introduced. Finally, two machine learning-based methods for time series anomaly detection are employed to detect the attacks. Matrix Profiles are employed more successfully than a predictor Long Short-Term Memory network, a class of neural networks.
Secure Routing in IoT: Evaluation of RPL Secure Mode under Attacks
Venue: arXiv cs.CR 2019
Year: 2019

As the Routing Protocol for Low Power and Lossy Networks (RPL) became the standard for routing in the Internet of Things (IoT) networks, many researchers had investigated the security aspects of this protocol. However, no work (to the best of our knowledge) has investigated the use of the security mechanisms included in the protocol standard, due to the fact that there was no implementation for these features in any IoT operating system yet. A partial implementation of RPL security mechanisms was presented recently for Contiki operating system (by Perazzo et al.), which provided us with the opportunity to examine RPL security mechanisms. In this paper, we investigate the effects and challenges of using RPL security mechanisms under common routing attacks. First, a comparison of RPL performance, with and without its security mechanisms, under three routing attacks (Blackhole, Selective- Forward, and Neighbor attacks) is conducted using several metrics (e.g., average data packet delivery rate, average data packet delay, average power consumption... etc.) Based on the observations from this comparison, we came with few suggestions that could reduce the effects of such attacks, without having added security mechanisms for RPL.
Decidability of an Expressive Description Logic with Rational Grading
Venue: arXiv cs.LO 2019
Year: 2019

In this paper syntactic objects---concept constructors called part restrictions which realize rational grading are considered in Description Logics (DLs). Being able to convey statements about a rational part of a set of successors, part restrictions essentially enrich the expressive capabilities of DLs. We examine an extension of well-studied DL ALCQIHR+ with part restrictions, and prove that the reasoning in the extended logic is still decidable. The proof uses tableaux technique augmented with indices technique, designed for dealing with part restrictions.
Tempus Volat, Hora Fugit -- A Survey of Tie-Oriented Dynamic Network Models in Discrete and Continuous Time
Venue: arXiv cs.SI 2019
Year: 2019

Given the growing number of available tools for modeling dynamic networks, the choice of a suitable model becomes central. The goal of this survey is to provide an overview of tie-oriented dynamic network models. The survey is focused on introducing binary network models with their corresponding assumptions, advantages, and shortfalls. The models are divided according to generating processes, operating in discrete and continuous time. First, we introduce the Temporal Exponential Random Graph Model (TERGM) and the Separable TERGM (STERGM), both being time-discrete models. These models are then contrasted with continuous process models, focusing on the Relational Event Model (REM). We additionally show how the REM can handle time-clustered observations, i.e., continuous time data observed at discrete time points. Besides the discussion of theoretical properties and fitting procedures, we specifically focus on the application of the models on two networks that represent international arms transfers and email exchange. The data allow to demonstrate the applicability and interpretation of the network models.
Additive Noise Annealing and Approximation Properties of Quantized Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

We present a theoretical and experimental investigation of the quantization problem for artificial neural networks. We provide a mathematical definition of quantized neural networks and analyze their approximation capabilities, showing in particular that any Lipschitz-continuous map defined on a hypercube can be uniformly approximated by a quantized neural network. We then focus on the regularization effect of additive noise on the arguments of multi-step functions inherent to the quantization of continuous variables. In particular, when the expectation operator is applied to a non-differentiable multi-step random function, and if the underlying probability density is differentiable (in either classical or weak sense), then a differentiable function is retrieved, with explicit bounds on its Lipschitz constant. Based on these results, we propose a novel gradient-based training algorithm for quantized neural networks that generalizes the straight-through estimator, acting on noise applied to the network's parameters. We evaluate our algorithm on the CIFAR-10 and ImageNet image classification benchmarks, showing state-of-the-art performance on AlexNet and MobileNetV2 for ternary networks.
A Polynomial-Based Approach for Architectural Design and Learning with Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

In this effort we propose a novel approach for reconstructing multivariate functions from training data, by identifying both a suitable network architecture and an initialization using polynomial-based approximations. Training deep neural networks using gradient descent can be interpreted as moving the set of network parameters along the loss landscape in order to minimize the loss functional. The initialization of parameters is important for iterative training methods based on descent. Our procedure produces a network whose initial state is a polynomial representation of the training data. The major advantage of this technique is from this initialized state the network may be improved using standard training procedures. Since the network already approximates the data, training is more likely to produce a set of parameters associated with a desirable local minimum. We provide the details of the theory necessary for constructing such networks and also consider several numerical examples that reveal our approach ultimately produces networks which can be effectively trained from our initialized state to achieve an improved approximation for a large class of target functions.
Debiasing Word Embeddings Improves Multimodal Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

In recent years, pretrained word embeddings have proved useful for multimodal neural machine translation (NMT) models to address the shortage of available datasets. However, the integration of pretrained word embeddings has not yet been explored extensively. Further, pretrained word embeddings in high dimensional spaces have been reported to suffer from the hubness problem. Although some debiasing techniques have been proposed to address this problem for other natural language processing tasks, they have seldom been studied for multimodal NMT models. In this study, we examine various kinds of word embeddings and introduce two debiasing techniques for three multimodal NMT models and two language pairs -- English-German translation and English-French translation. With our optimal settings, the overall performance of multimodal models was improved by up to +1.93 BLEU and +2.02 METEOR for English-German translation and +1.73 BLEU and +0.95 METEOR for English-French translation.
Multi-Model Investigative Exploration of Social Media Data with boutique: A Case Study in Public Health
Venue: arXiv cs.DB 2019
Year: 2019

We present our experience with a data science problem in Public Health, where researchers use social media (Twitter) to determine whether the public shows awareness of HIV prevention measures offered by Public Health campaigns. To help the researcher, we develop an investigative exploration system called boutique that allows a user to perform a multi-step visualization and exploration of data through a dashboard interface. Unique features of boutique includes its ability to handle heterogeneous types of data provided by a polystore, and its ability to use computation as part of the investigative exploration process. In this paper, we present the design of the boutique middleware and walk through an investigation process for a real-life problem.
Constructing vectorial bent functions via second-order derivatives
Venue: arXiv cs.IT 2019
Year: 2019

Let $n$ be an even positive integer, and $m<n$ be one of its positive divisors. In this paper, inspired by a nice work of Tang et al. on constructing large classes of bent functions from known bent functions [27, IEEE TIT, 63(10): 6149-6157, 2017], we consider the construction of vectorial bent and vectorial plateaued $(n,m)$-functions of the form $H(x)=G(x)+g(x)$, where $G(x)$ is a vectorial bent $(n,m)$-function, and $g(x)$ is a Boolean function over $\mathbb{F}_{2^{n}}$. We find an efficient generic method to construct vectorial bent and vectorial plateaued functions of this form by establishing a link between the condition on the second-order derivatives and the key condition given by [27]. This allows us to provide (at least) three new infinite families of vectorial bent functions with high algebraic degrees. New vectorial plateaued $(n,m+t)$-functions are also obtained ($t\geq 0$ depending on $n$ can be taken as a very large number), two classes of which have the maximal number of bent components.
Enhancing Adversarial Defense by k-Winners-Take-All
Venue: arXiv cs.LG 2019
Year: 2019

We propose a simple change to existing neural network structures for better defending against gradient-based adversarial attacks. Instead of using popular activation functions (such as ReLU), we advocate the use of k-Winners-Take-All (k-WTA) activation, a C0 discontinuous function that purposely invalidates the neural network model's gradient at densely distributed input data points. The proposed k-WTA activation can be readily used in nearly all existing networks and training methods with no significant overhead. Our proposal is theoretically rationalized. We analyze why the discontinuities in k-WTA networks can largely prevent gradient-based search of adversarial examples and why they at the same time remain innocuous to the network training. This understanding is also empirically backed. We test k-WTA activation on various network structures optimized by a training method, be it adversarial training or not. In all cases, the robustness of k-WTA networks outperforms that of traditional networks under white-box attacks.
Bivariate Beta-LSTM
Venue: arXiv cs.LG 2019
Year: 2019

Long Short-Term Memory (LSTM) infers the long term dependency through a cell state maintained by the input and the forget gate structures, which models a gate output as a value in [0,1] through a sigmoid function. However, due to the graduality of the sigmoid function, the sigmoid gate is not flexible in representing multi-modality or skewness. Besides, the previous models lack modeling on the correlation between the gates, which would be a new method to adopt inductive bias for a relationship between previous and current input. This paper proposes a new gate structure with the bivariate Beta distribution. The proposed gate structure enables probabilistic modeling on the gates within the LSTM cell so that the modelers can customize the cell state flow with priors and distributions. Moreover, we theoretically show the higher upper bound of the gradient compared to the sigmoid function, and we empirically observed that the bivariate Beta distribution gate structure provides higher gradient values in training. We demonstrate the effectiveness of bivariate Beta gate structure on the sentence classification, image classification, polyphonic music modeling, and image caption generation.
Leveraging Domain Knowledge to Improve Microscopy Image Segmentation with Lifted Multicuts
Venue: arXiv cs.CV 2019
Year: 2019

The throughput of electron microscopes has increased significantly in recent years, enabling detailed analysis of cell morphology and ultrastructure. Analysis of neural circuits at single-synapse resolution remains the flagship target of this technique, but applications to cell and developmental biology are also starting to emerge at scale. The amount of data acquired in such studies makes manual instance segmentation, a fundamental step in many analysis pipelines, impossible. While automatic segmentation approaches have improved significantly thanks to the adoption of convolutional neural networks, their accuracy still lags behind human annotations and requires additional manual proof-reading. A major hindrance to further improvements is the limited field of view of the segmentation networks preventing them from exploiting the expected cell morphology or other prior biological knowledge which humans use to inform their segmentation decisions. In this contribution, we show how such domain-specific information can be leveraged by expressing it as long-range interactions in a graph partitioning problem known as the lifted multicut problem. Using this formulation, we demonstrate significant improvement in segmentation accuracy for three challenging EM segmentation problems from neuroscience and cell biology.
Learning and Composing Primitive Skills for Dual-arm Manipulation
Venue: arXiv cs.RO 2019
Year: 2019

In an attempt to confer robots with complex manipulation capabilities, dual-arm anthropomorphic systems have become an important research topic in the robotics community. Most approaches in the literature rely upon a great understanding of the dynamics underlying the system's behaviour and yet offer limited autonomous generalisation capabilities. To address these limitations, this work proposes a modelisation for dual-arm manipulators based on dynamic movement primitives laying in two orthogonal spaces. The modularity and learning capabilities of this model are leveraged to formulate a novel end-to-end learning-based framework which (i) learns a library of primitive skills from human demonstrations, and (ii) composes such knowledge simultaneously and sequentially to confront novel scenarios. The feasibility of the proposal is evaluated by teaching the iCub humanoid the basic skills to succeed on simulated dual-arm pick-and-place tasks. The results suggest the learning and generalisation capabilities of the proposed framework extend to autonomously conduct undemonstrated dual-arm manipulation tasks.