OpenRaiser/Intern-Atlas huggingface dataset:
PARQR: Augmenting the Piazza Online Forum to Better Support Degree Seeking Online Masters Students
Venue: arXiv cs.HC 2019
Year: 2019

We introduce PARQR, a tool for online education forums that reduces duplicate posts by 40\% in a degree seeking online masters program at a top university. Instead of performing a standard keyword search, PARQR monitors questions as students compose them and continuously suggests relevant posts. In testing, PARQR correctly recommends a relevant post, if one exists, 73.5\% of the time. We discuss PARQR's design, initial experimental results comparing different semesters with and without PARQR, and interviews we conducted with teaching instructors regarding their experience with PARQR.
Correct, Fast Remote Persistence
Venue: arXiv cs.DC 2019
Year: 2019

Persistence of updates to remote byte-addressable persistent memory (PM), using RDMA operations (RDMA updates), is a poorly understood subject. Visibility of RDMA updates on the remote server is not the same as persistence of those updates. The remote server's configuration has significant implications on what it means for RDMA updates to be persistent on the remote server. This leads to significant implications on methods needed to correctly persist those updates. This paper presents a comprehensive taxonomy of system configurations and the corresponding methods to ensure correct remote persistence of RDMA updates. We show that the methods for correct, fast remote persistence vary dramatically, with corresponding performance trade offs, between different remote server configurations.
Auxiliary Learning for Deep Multi-task Learning
Venue: arXiv cs.CV 2019
Year: 2019

Multi-task learning (MTL) is an efficient solution to solve multiple tasks simultaneously in order to get better speed and performance than handling each single-task in turn. The most current methods can be categorized as either: (i) hard parameter sharing where a subset of the parameters is shared among tasks while other parameters are task-specific; or (ii) soft parameter sharing where all parameters are task-specific but they are jointly regularized. Both methods suffer from limitations: the shared hidden layers of the former are difficult to optimize due to the competing objectives while the complexity of the latter grows linearly with the increasing number of tasks. To mitigate those drawbacks, this paper proposes an alternative, where we explicitly construct an auxiliary module to mimic the soft parameter sharing for assisting the optimization of the hard parameter sharing layers in the training phase. In particular, the auxiliary module takes the outputs of the shared hidden layers as inputs and is supervised by the auxiliary task loss. During training, the auxiliary module is jointly optimized with the MTL network, serving as a regularization by introducing an inductive bias to the shared layers. In the testing phase, only the original MTL network is kept. Thus our method avoids the limitation of both categories. We evaluate the proposed auxiliary module on pixel-wise prediction tasks, including semantic segmentation, depth estimation, and surface normal prediction with different network structures. The extensive experiments over various settings verify the effectiveness of our methods.
Effective Domain Knowledge Transfer with Soft Fine-tuning
Venue: arXiv cs.CV 2019
Year: 2019

Convolutional neural networks require numerous data for training. Considering the difficulties in data collection and labeling in some specific tasks, existing approaches generally use models pre-trained on a large source domain (e.g. ImageNet), and then fine-tune them on these tasks. However, the datasets from source domain are simply discarded in the fine-tuning process. We argue that the source datasets could be better utilized and benefit fine-tuning. This paper firstly introduces the concept of general discrimination to describe ability of a network to distinguish untrained patterns, and then experimentally demonstrates that general discrimination could potentially enhance the total discrimination ability on target domain. Furthermore, we propose a novel and light-weighted method, namely soft fine-tuning. Unlike traditional fine-tuning which directly replaces optimization objective by a loss function on the target domain, soft fine-tuning effectively keeps general discrimination by holding the previous loss and removes it softly. By doing so, soft fine-tuning improves the robustness of the network to data bias, and meanwhile accelerates the convergence. We evaluate our approach on several visual recognition tasks. Extensive experimental results support that soft fine-tuning provides consistent improvement on all evaluated tasks, and outperforms the state-of-the-art significantly. Codes will be made available to the public.
Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection
Venue: arXiv cs.CV 2019
Year: 2019

Recently, Neural Architecture Search has achieved great success in large-scale image classification. In contrast, there have been limited works focusing on architecture search for object detection, mainly because the costly ImageNet pre-training is always required for detectors. Training from scratch, as a substitute, demands more epochs to converge and brings no computation saving. To overcome this obstacle, we introduce a practical neural architecture transformation search(NATS)algorithm for object detection in this paper. Instead of searching and constructing an entire network, NATS explores the architecture space on the base of existing network and reusing its weights. We propose a novel neural architecture search strategy in channel-level instead of path-level and devise a search space specially targeting at object detection. With the combination of these two designs, an architecture transformation scheme could be discovered to adapt a network designed for image classification to task of object detection. Since our method is gradient-based and only searches for a transformation scheme, the weights of models pretrained inImageNet could be utilized in both searching and retraining stage, which makes the whole process very efficient. The transformed network requires no extra parameters and FLOPs, and is friendly to hardware optimization, which is practical to use in real-time application. In experiments, we demonstrate the effectiveness of NATSon networks like ResNet and ResNeXt. Our transformed networks, combined with various detection frameworks, achieve significant improvements on the COCO dataset while keeping fast.
SwarmTouch: Guiding a Swarm of Micro-Quadrotors with Impedance Control using a Wearable Tactile Interface
Venue: arXiv cs.RO 2019
Year: 2019

To achieve a smooth and safe guiding of a drone formation by a human operator, we propose a novel interaction strategy for a human-swarm communication which combines impedance control and vibrotactile feedback. The presented approach takes into account the human hand velocity and changes the formation shape and dynamics accordingly using impedance interlinks simulated between quadrotors, which helps to achieve a natural swarm behavior. Several tactile patterns representing static and dynamic parameters of the swarm are proposed. The user feels the state of the swarm at the fingertips and receives valuable information to improve the controllability of the complex formation. A user study revealed the patterns with high recognition rates. A flight experiment demonstrated the possibility to accurately navigate the formation in a cluttered environment using only tactile feedback. Subjects stated that tactile sensation allows guiding the drone formation through obstacles and makes the human-swarm communication more interactive. The proposed technology can potentially have a strong impact on the human-swarm interaction, providing a higher level of awareness during the swarm navigation.
Kernel absolute summability is only sufficient for RKHS stability
Venue: arXiv cs.LG 2019
Year: 2019

Regularized approaches have been successfully applied to linear system identification in recent years. Many of them model unknown impulse responses exploiting the so called Reproducing Kernel Hilbert spaces (RKHSs) that enjoy the notable property of being in one-to-one correspondence with the class of positive semidefinite kernels. The necessary and sufficient condition for a RKHS to be stable, i.e. to contain only BIBO stable linear dynamic systems, has been known in the literature at least since 2006. However, an open question still persists and concerns the equivalence of such condition with the absolute summability of the kernel. This paper provides a definite answer to this matter by proving that such correspondence does not hold. A counterexample is introduced that illustrates the existence of stable RKHSs that are induced by non-absolutely summable kernels.
A Transfer Learning Approach for Network Intrusion Detection
Venue: arXiv cs.LG 2019
Year: 2019

Convolution Neural Network (ConvNet) offers a high potential to generalize input data. It has been widely used in many application areas, such as visual imagery, where comprehensive learning datasets are available and a ConvNet model can be well trained and perform the required function effectively. ConvNet can also be applied to network intrusion detection. However, the currently available datasets related to the network intrusion are often inadequate, which makes the ConvNet learning deficient, hence the trained model is not competent in detecting unknown intrusions. In this paper, we propose a ConvNet model using transfer learning for network intrusion detection. The model consists of two concatenated ConvNets and is built on a two-stage learning process: learning a base dataset and transferring the learned knowledge to the learning of the target dataset. Our experiments on the NSL-KDD dataset show that the proposed model can improve the detection accuracy not only on the test dataset containing mostly known attacks (KDDTest+) but also on the test dataset featuring many novel attacks (KDDTest-21) -- about 2.68\% improvement on KDDTest+ and 22.02\% on KDDTest-21 can be achieved, as compared to the traditional ConvNet model.
Modelling transport provision in a polycentric mega city region
Venue: arXiv cs.MA 2019
Year: 2019

The aim of this paper is to present a model of interaction between transport and land use which aims at endogenously integrates the provision of transportation infrastructure and its effects on land use, with a long term perspective (Lowry, 1964, Wegener, 2004, Levinson, 2011, Bretagnolle, 2014, Mimeur et al., 2015). LUTECIA (Land Use, Transport, Evaluation of Cooperation, Infrastructure provision and Agglomeration effects) model puts emphasis on multiscale processes of urban growth, in the context of the emergence of Mega-City Regions (MCR, Hall & Pain, 2006). It allows us, in this exploratory phase, to characterize via simulation the conditions of development of polycentric metropolises. Nevertheless, we argue that such approaches are all the more necessary that we are in a period of multiple transitions that classical modelling tools have difficulty to capture.
A Low-Memory Time-Efficient Implementation of Outermorphisms for Higher-Dimensional Geometric Algebras
Venue: arXiv cs.MS 2019
Year: 2019

From the beginning of David Hestenes rediscovery of geometric algebra in the 1960s, outermorphisms have been a cornerstone in the mathematical development of GA. Many important mathematical formulations in GA can be expressed as outermorphisms such as versor products, linear projection operators, and mapping between related coordinate frames. Over the last two decades, GA-based mathematical models and software implementations have been developed in many fields of science and engineering. As such, efficient implementations of outermorphisms are of significant importance within this context. This work attempts to shed some light on the problem of optimizing software implementations of outermorphisms for practical prototyping applications using geometric algebra. The approach we propose here for implementing outermorphisms requires orders of magnitude less memory compared to other common approaches, while being comparable in time performance, especially for high-dimensional geometric algebras.
Swap Stability in Schelling Games on Graphs
Venue: arXiv cs.GT 2019
Year: 2019

We study a recently introduced class of strategic games that is motivated by and generalizes Schelling's well-known residential segregation model. These games are played on undirected graphs, with the set of agents partitioned into multiple types; each agent either occupies a node of the graph and never moves away or aims to maximize the fraction of her neighbors who are of her own type. We consider a variant of this model that we call swap Schelling games, where the number of agents is equal to the number of nodes of the graph, and agents may {\em swap} positions with other agents to increase their utility. We study the existence, computational complexity and quality of equilibrium assignments in these games, both from a social welfare perspective and from a diversity perspective.
Are Adversarial Robustness and Common Perturbation Robustness Independent Attributes ?
Venue: arXiv cs.LG 2019
Year: 2019

Neural Networks have been shown to be sensitive to common perturbations such as blur, Gaussian noise, rotations, etc. They are also vulnerable to some artificial malicious corruptions called adversarial examples. The adversarial examples study has recently become very popular and it sometimes even reduces the term "adversarial robustness" to the term "robustness". Yet, we do not know to what extent the adversarial robustness is related to the global robustness. Similarly, we do not know if a robustness to various common perturbations such as translations or contrast losses for instance, could help with adversarial corruptions. We intend to study the links between the robustnesses of neural networks to both perturbations. With our experiments, we provide one of the first benchmark designed to estimate the robustness of neural networks to common perturbations. We show that increasing the robustness to carefully selected common perturbations, can make neural networks more robust to unseen common perturbations. We also prove that adversarial robustness and robustness to common perturbations are independent. Our results make us believe that neural network robustness should be addressed in a broader sense.
Lower bound performances for average consensus in open multi-agent systems (extended version)
Venue: arXiv cs.MA 2019
Year: 2019

We derive fundamental limitations on the performances of intrinsic averaging algorithms in open multi-agent systems, which are systems subject to random arrivals and departures of agents. Each agent holds a value, and their goal is to estimate the average of the values of the agents presently in the system. We provide a lower bound on the expected Mean Square Error for any estimation algorithm, assuming that the number of agents remains constant and that communications are random and pairwise. Our derivation is based on the expected error obtained with an optimal algorithm under conditions more favorable than those the actual problem allows, and relies on an analysis of the constraints on the information spreading mechanisms in the system, and relaxations of these.
Neuromorphic Visual Odometry System for Intelligent Vehicle Application with Bio-inspired Vision Sensor
Venue: arXiv cs.RO 2019
Year: 2019

The neuromorphic camera is a brand new vision sensor that has emerged in recent years. In contrast to the conventional frame-based camera, the neuromorphic camera only transmits local pixel-level changes at the time of its occurrence and provides an asynchronous event stream with low latency. It has the advantages of extremely low signal delay, low transmission bandwidth requirements, rich information of edges, high dynamic range etc., which make it a promising sensor in the application of in-vehicle visual odometry system. This paper proposes a neuromorphic in-vehicle visual odometry system using feature tracking algorithm. To the best of our knowledge, this is the first in-vehicle visual odometry system that only uses a neuromorphic camera, and its performance test is carried out on actual driving datasets. In addition, an in-depth analysis of the results of the experiment is provided. The work of this paper verifies the feasibility of in-vehicle visual odometry system using neuromorphic cameras.
A Generalized Web Component for Domain-Independent Smart Assistants
Venue: arXiv cs.HC 2019
Year: 2019

This article introduces an open-source web component, Instant Expert, which allows robust and efficient integration of a natural language question answering system to web-based platforms in any domain. Web Components are a set of web technologies to allow the creation of reusable, customizable, and encapsulated HTML elements. The Instant Expert web component consists of the user input (i.e. text, voice, multi-selection), question processing, and user interface modules. Two use cases are developed to demonstrate the component's features, benefits, and usage. The goal of this project is to pave the way for next-generation information systems by mitigating the challenges of developing voice-enabled and domain-informed smart assistants for communicating knowledge in any domain.
On the discriminative power of Hyper-parameters in Cross-Validation and how to choose them
Venue: arXiv cs.IR 2019
Year: 2019

Hyper-parameters tuning is a crucial task to make a model perform at its best. However, despite the well-established methodologies, some aspects of the tuning remain unexplored. As an example, it may affect not just accuracy but also novelty as well as it may depend on the adopted dataset. Moreover, sometimes it could be sufficient to concentrate on a single parameter only (or a few of them) instead of their overall set. In this paper we report on our investigation on hyper-parameters tuning by performing an extensive 10-Folds Cross-Validation on MovieLens and Amazon Movies for three well-known baselines: User-kNN, Item-kNN, BPR-MF. We adopted a grid search strategy considering approximately 15 values for each parameter, and we then evaluated each combination of parameters in terms of accuracy and novelty. We investigated the discriminative power of nDCG, Precision, Recall, MRR, EFD, EPC, and, finally, we analyzed the role of parameters on model evaluation for Cross-Validation.
Causal Effects of Brevity on Style and Success in Social Media
Venue: arXiv cs.SI 2019
Year: 2019

In online communities, where billions of people strive to propagate their messages, understanding how wording affects success is of primary importance. In this work, we are interested in one particularly salient aspect of wording: brevity. What is the causal effect of brevity on message success? What are the linguistic traits of brevity? When is brevity beneficial, and when is it not? Whereas most prior work has studied the effect of wording on style and success in observational setups, we conduct a controlled experiment, in which crowd workers shorten social media posts to prescribed target lengths and other crowd workers subsequently rate the original and shortened versions. This allows us to isolate the causal effect of brevity on the success of a message. We find that concise messages are on average more successful than the original messages up to a length reduction of 30-40%. The optimal reduction is on average between 10% and 20%. The observed effect is robust across different subpopulations of raters and is the strongest for raters who visit social media on a daily basis. Finally, we discover unique linguistic and content traits of brevity and correlate them with the measured probability of success in order to distinguish effective from ineffective shortening strategies. Overall, our findings are important for developing a better understanding of the effect of brevity on the success of messages in online social media.
Additive function approximation in the brain
Venue: arXiv cs.NE 2019
Year: 2019

Many biological learning systems such as the mushroom body, hippocampus, and cerebellum are built from sparsely connected networks of neurons. For a new understanding of such networks, we study the function spaces induced by sparse random features and characterize what functions may and may not be learned. A network with $d$ inputs per neuron is found to be equivalent to an additive model of order $d$, whereas with a degree distribution the network combines additive terms of different orders. We identify three specific advantages of sparsity: additive function approximation is a powerful inductive bias that limits the curse of dimensionality, sparse networks are stable to outlier noise in the inputs, and sparse random features are scalable. Thus, even simple brain architectures can be powerful function approximators. Finally, we hope that this work helps popularize kernel theories of networks among computational neuroscientists.
An Empirical Study on the Characteristics of Question-Answering Process on Developer Forums
Venue: arXiv cs.SE 2019
Year: 2019

Developer forums are one of the most popular and useful Q&A websites on API usages. The analysis of API forums can be a critical resource for the automated question and answer approaches. In this paper, we empirically study three API forums including Twitter, eBay, and AdWords, to investigate the characteristics of question-answering process. We observe that +60% of the posts on all three forums were answered by providing API method names or documentation. +85% of the questions were answered by API development teams and the answers from API development teams drew fewer follow-up questions. Our results provide empirical evidences for us in a future work to build automated solutions to answer developer questions on API forums.
Coarse2Fine: A Two-stage Training Method for Fine-grained Visual Classification
Venue: arXiv cs.CV 2019
Year: 2019

Small inter-class and large intra-class variations are the main challenges in fine-grained visual classification. Objects from different classes share visually similar structures and objects in the same class can have different poses and viewpoints. Therefore, the proper extraction of discriminative local features (e.g. bird's beak or car's headlight) is crucial. Most of the recent successes on this problem are based upon the attention models which can localize and attend the local discriminative objects parts. In this work, we propose a training method for visual attention networks, Coarse2Fine, which creates a differentiable path from the input space to the attended feature maps. Coarse2Fine learns an inverse mapping function from the attended feature maps to the informative regions in the raw image, which will guide the attention maps to better attend the fine-grained features. We show Coarse2Fine and orthogonal initialization of the attention weights can surpass the state-of-the-art accuracies on common fine-grained classification tasks.
Port-Hamiltonian Approach to Neural Network Training
Venue: arXiv cs.NE 2019
Year: 2019

Neural networks are discrete entities: subdivided into discrete layers and parametrized by weights which are iteratively optimized via difference equations. Recent work proposes networks with layer outputs which are no longer quantized but are solutions of an ordinary differential equation (ODE); however, these networks are still optimized via discrete methods (e.g. gradient descent). In this paper, we explore a different direction: namely, we propose a novel framework for learning in which the parameters themselves are solutions of ODEs. By viewing the optimization process as the evolution of a port-Hamiltonian system, we can ensure convergence to a minimum of the objective function. Numerical experiments have been performed to show the validity and effectiveness of the proposed methods.