OpenRaiser/Intern-Atlas huggingface dataset:
A Solution to Time-Varying Markov Decision Processes
Venue: arXiv cs.RO 2018
Year: 2018

We consider a decision-making problem where the environment varies both in space and time. Such problems arise naturally when considering e.g., the navigation of an underwater robot amidst ocean currents or the navigation of an aerial vehicle in wind. To model such spatiotemporal variation, we extend the standard Markov Decision Process (MDP) to a new framework called the Time-Varying Markov Decision Process (TVMDP). The TVMDP has a time-varying state transition model and transforms the standard MDP that considers only immediate and static uncertainty descriptions of state transitions, to a framework that is able to adapt to future time-varying transition dynamics over some horizon. We show how to solve a TVMDP via a redesign of the MDP value propagation mechanisms by incorporating the introduced dynamics along the temporal dimension. We validate our framework in a marine robotics navigation setting using spatiotemporal ocean data and show that it outperforms prior efforts.
Transport Analysis of Infinitely Deep Neural Network
Venue: arXiv cs.LG 2018
Year: 2018

We investigated the feature map inside deep neural networks (DNNs) by tracking the transport map. We are interested in the role of depth (why do DNNs perform better than shallow models?) and the interpretation of DNNs (what do intermediate layers do?) Despite the rapid development in their application, DNNs remain analytically unexplained because the hidden layers are nested and the parameters are not faithful. Inspired by the integral representation of shallow NNs, which is the continuum limit of the width, or the hidden unit number, we developed the flow representation and transport analysis of DNNs. The flow representation is the continuum limit of the depth or the hidden layer number, and it is specified by an ordinary differential equation with a vector field. We interpret an ordinary DNN as a transport map or a Euler broken line approximation of the flow. Technically speaking, a dynamical system is a natural model for the nested feature maps. In addition, it opens a new way to the coordinate-free treatment of DNNs by avoiding the redundant parametrization of DNNs. Following Wasserstein geometry, we analyze a flow in three aspects: dynamical system, continuity equation, and Wasserstein gradient flow. A key finding is that we specified a series of transport maps of the denoising autoencoder (DAE). Starting from the shallow DAE, this paper develops three topics: the transport map of the deep DAE, the equivalence between the stacked DAE and the composition of DAEs, and the development of the double continuum limit or the integral representation of the flow representation. As partial answers to the research questions, we found that deeper DAEs converge faster and the extracted features are better; in addition, a deep Gaussian DAE transports mass to decrease the Shannon entropy of the data distribution.
Multicores-periphery structure in networks
Venue: arXiv cs.SI 2018
Year: 2018

Many real-world networks exhibit a multicores-periphery structure, with densely connected vertices in multiple cores surrounded by a general periphery of sparsely connected vertices. Identification of the multicores-periphery structure can provide a new lens to understand the structures and functions of various real-world networks. This paper defines the multicores-periphery structure and introduces an algorithm to identify the optimal partition of multiple cores and the periphery in general networks. We demonstrate the performance of our algorithm by applying it to a well-known social network and a patent technology network, which are best characterized by the multicores-periphery structure. The analyses also reveal the differences between our multicores-periphery detection algorithm and two state-of-the-art algorithms for detecting the single core-periphery structure and community structure.
Factors and actors leading to the adoption of a JavaScript framework
Venue: arXiv cs.SE 2018
Year: 2018

The increasing popularity of JavaScript has led to a variety of JavaScript frameworks that aim to help developers to address programming tasks. However, the number of JavaScript frameworks has risen rapidly to thousands of versions. It is challenging for practitioners to identify the frameworks that best fit their needs and to develop new ones which fit such needs. Furthermore, there is a lack of knowledge regarding what drives developers towards the choice. This paper explores the factors and actors that lead to the choice of a JavaScript framework. We conducted a qualitative interpretive study of semi-structured interviews. We interviewed 18 decision makers regarding the JavaScript framework selection, up to reaching theoretical saturation. Through coding the interview responses, we offer a model of desirable JavaScript framework adoption factors. The factors are grouped into categories that are derived via the Unified Theory of Acceptance and Use of Technology. The factors are performance expectancy (performance, size), effort expectancy (automatization, learnability, complexity, understandability), social influence (competitor analysis, collegial advice, community size, community responsiveness), facilitating conditions (suitability, updates, modularity, isolation, extensibility), and price value. A combination of four actors, which are customer, developer, team, and team leader, leads to the choice. Our model contributes to the body of knowledge related to the adoption of technology by software engineers. As a practical implication, our model is useful for decision makers when evaluating JavaScript frameworks, as well as for developers for producing desirable frameworks.
Word2Vec is a special case of Kernel Correspondence Analysis and Kernels for Natural Language Processing
Venue: arXiv cs.LG 2018
Year: 2018

We show that correspondence analysis (CA) is equivalent to defining a Gini index with appropriately scaled one-hot encoding. Using this relation, we introduce a nonlinear kernel extension to CA. This extended CA gives a known analysis for natural language via specialized kernels that use an appropriate contingency table. We propose a semi-supervised CA, which is a special case of the kernel extension to CA. Because CA requires excessive memory if applied to numerous categories, CA has not been used for natural language processing. We address this problem by introducing delayed evaluation to randomized singular value decomposition. The memory-efficient CA is then applied to a word-vector representation task. We propose a tail-cut kernel, which is an extension to the skip-gram within the kernel extension to CA. Our tail-cut kernel outperforms existing word-vector representation methods.
Local communities obstruct global consensus: Naming game on multi-local-world networks
Venue: arXiv cs.SI 2018
Year: 2018

Community structure is essential for social communications, where individuals belonging to the same community are much more actively interacting and communicating with each other than those in different communities within the human society. Naming game, on the other hand, is a social communication model that simulates the process of learning a name of an object within a community of humans, where the individuals can generally reach global consensus asymptotically through iterative pair-wise conversations. The underlying network indicates the relationships among the individuals. In this paper, three typical topologies, namely random-graph, small-world and scale-free networks, are employed, which are embedded with the multi-local-world community structure, to study the naming game. Simulations show that 1) the convergence process to global consensus is getting slower as the community structure becomes more prominent, and eventually might fail; 2) if the inter-community connections are sufficiently dense, neither the number nor the size of the communities affects the convergence process; and 3) for different topologies with the same average node-degree, local clustering of individuals obstruct or prohibit global consensus to take place. The results reveal the role of local communities in a global naming game in social network studies.
Self Paced Deep Learning for Weakly Supervised Object Detection
Venue: arXiv cs.CV 2018
Year: 2018

In a weakly-supervised scenario object detectors need to be trained using image-level annotation alone. Since bounding-box-level ground truth is not available, most of the solutions proposed so far are based on an iterative, Multiple Instance Learning framework in which the current classifier is used to select the highest-confidence boxes in each image, which are treated as pseudo-ground truth in the next training iteration. However, the errors of an immature classifier can make the process drift, usually introducing many of false positives in the training dataset. To alleviate this problem, we propose in this paper a training protocol based on the self-paced learning paradigm. The main idea is to iteratively select a subset of images and boxes that are the most reliable, and use them for training. While in the past few years similar strategies have been adopted for SVMs and other classifiers, we are the first showing that a self-paced approach can be used with deep-network-based classifiers in an end-to-end training pipeline. The method we propose is built on the fully-supervised Fast-RCNN architecture and can be applied to similar architectures which represent the input image as a bag of boxes. We show state-of-the-art results on Pascal VOC 2007, Pascal VOC 2010 and ILSVRC 2013. On ILSVRC 2013 our results based on a low-capacity AlexNet network outperform even those weakly-supervised approaches which are based on much higher-capacity networks.
Action Classification via Concepts and Attributes
Venue: arXiv cs.CV 2018
Year: 2018

Classes in natural images tend to follow long tail distributions. This is problematic when there are insufficient training examples for rare classes. This effect is emphasized in compound classes, involving the conjunction of several concepts, such as those appearing in action-recognition datasets. In this paper, we propose to address this issue by learning how to utilize common visual concepts which are readily available. We detect the presence of prominent concepts in images and use them to infer the target labels instead of using visual features directly, combining tools from vision and natural-language processing. We validate our method on the recently introduced HICO dataset reaching a mAP of 31.54\% and on the Stanford-40 Actions dataset, where the proposed method outperforms that obtained by direct visual features, obtaining an accuracy 83.12\%. Moreover, the method provides for each class a semantically meaningful list of keywords and relevant image regions relating it to its constituent concepts.
Hitting minors, subdivisions, and immersions in tournaments
Venue: arXiv cs.DM 2018
Year: 2018

The Erdős-Pósa property relates parameters of covering and packing of combinatorial structures and has been mostly studied in the setting of undirected graphs. In this note, we use results of Chudnovsky, Fradkin, Kim, and Seymour to show that, for every directed graph $H$ (resp. strongly-connected directed graph $H$), the class of directed graphs that contain $H$ as a strong minor (resp. butterfly minor, topological minor) has the vertex-Erdős-Pósa property in the class of tournaments. We also prove that if $H$ is a strongly-connected directed graph, the class of directed graphs containing $H$ as an immersion has the edge-Erdős-Pósa property in the class of tournaments.
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Object recognition is an important task for improving the ability of visual systems to perform complex scene understanding. Recently, the Exponential Linear Unit (ELU) has been proposed as a key component for managing bias shift in Convolutional Neural Networks (CNNs), but defines a parameter that must be set by hand. In this paper, we propose learning a parameterization of ELU in order to learn the proper activation shape at each layer in the CNNs. Our results on the MNIST, CIFAR-10/100 and ImageNet datasets using the NiN, Overfeat, All-CNN and ResNet networks indicate that our proposed Parametric ELU (PELU) has better performances than the non-parametric ELU. We have observed as much as a 7.28% relative error improvement on ImageNet with the NiN network, with only 0.0003% parameter increase. Our visual examination of the non-linear behaviors adopted by Vgg using PELU shows that the network took advantage of the added flexibility by learning different activations at different layers.
Task Assignment on Spatial Crowdsourcing [Experiments and Analyses] (Technical Report)
Venue: arXiv cs.DB 2018
Year: 2018

Recently, with the rapid development of mobile devices and the crowdsourcing platforms, the spatial crowdsourcing has attracted much attention from the database community. Specifically, spatial crowdsourcing refers to sending a location-based request to workers according to their positions, and workers need to physically move to specified locations to conduct tasks. Many works have studied task assignment problems in spatial crowdsourcing, however, their problem settings are different from each other. Thus, it is hard to compare the performances of existing algorithms on task assignment in spatial crowdsourcing. In this paper, we present a comprehensive experimental comparison of most existing algorithms on task assignment in spatial crowdsourcing. Specifically, we first give general definitions about spatial workers and spatial tasks based on definitions in the existing works such that the existing algorithms can be applied on the same synthetic and real data sets. Then, we provide an uniform implementation for all the tested algorithms of task assignment problems in spatial crowdsourcing (open sourced). Finally, based on the results on both synthetic and real data sets, we discuss the strengths and weaknesses of tested algorithms, which can guide future research on the same area and practical implementations of spatial crowdsourcing systems.
Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
Venue: arXiv cs.CL 2018
Year: 2018

In recent years great success has been achieved in sentiment classification for English, thanks in part to the availability of copious annotated resources. Unfortunately, most languages do not enjoy such an abundance of labeled data. To tackle the sentiment classification problem in low-resource languages without adequate annotated data, we propose an Adversarial Deep Averaging Network (ADAN) to transfer the knowledge learned from labeled data on a resource-rich source language to low-resource languages where only unlabeled data exists. ADAN has two discriminative branches: a sentiment classifier and an adversarial language discriminator. Both branches take input from a shared feature extractor to learn hidden representations that are simultaneously indicative for the classification task and invariant across languages. Experiments on Chinese and Arabic sentiment classification demonstrate that ADAN significantly outperforms state-of-the-art systems.
Discrete Calculus of Finite Sequences
Venue: arXiv cs.DM 2018
Year: 2018

The calculus of finite differences is a solid foundation for the development of operations such as the derivative and the integral for infinite sequences. Here we showed a way to extend it for finite sequences. We could then define convexity for finite sequences and some related concepts. To finalize, we propose a way to go from our extension to the calculus of finite differences.
Degrees of Freedom of Cache-Aided Wireless Interference Networks
Venue: arXiv cs.IT 2018
Year: 2018

We study the role of caches in wireless interference networks. We focus on content caching and delivery across a Gaussian interference network, where both transmitters and receivers are equipped with caches. We provide a constant-factor approximation of the system's degrees of freedom (DoF), for arbitrary number of transmitters, number of receivers, content library size, receiver cache size, and transmitter cache size (as long as the transmitters combined can store the entire content library among them). We demonstrate approximate optimality with respect to information-theoretic bounds that do not impose any restrictions on the caching and delivery strategies. Our characterization reveals three key insights. First, the approximate DoF is achieved using a strategy that separates the physical and network layers. This separation architecture is thus approximately optimal. Second, we show that increasing transmitter cache memory beyond what is needed to exactly store the entire library between all transmitters does not provide more than a constant-factor benefit to the DoF. A consequence is that transmit zero-forcing is not needed for approximate optimality. Third, we derive an interesting trade-off between the receiver memory and the number of transmitters needed for approximately maximal performance. In particular, if each receiver can store a constant fraction of the content library, then only a constant number of transmitters are needed. Our solution to the caching problem requires formulating and solving a new communication problem, the symmetric multiple multicast X-channel, for which we provide an exact DoF characterization.
Dynamic physical layer equalization in optical communication networks
Venue: arXiv cs.IT 2018
Year: 2018

In optical transport networks, signal lightpaths between two terminal nodes can be different due to current network conditions. Thus the transmission distance and accumulated dispersion in the lightpath cannot be predicted. Therefore, the adaptive compensation of dynamic dispersion is necessary in such networks to enable flexible routing and switching. In this paper, we present a detailed analysis on the adaptive dispersion compensation using the least-mean-square (LMS) algorithm in coherent optical communication networks. It is found that the variable-step-size LMS equalizer can achieve the same performance with a lower complexity, compared to the traditional LMS algorithm.
Revisiting the Majority Problem: Average-Case Analysis with Arbitrarily Many Colours
Venue: arXiv cs.DS 2018
Year: 2018

The majority problem is a special case of the heavy hitters problem. Given a collection of coloured balls, the task is to identify the majority colour or state that no such colour exists. Whilst the special case of two-colours has been well studied, the average-case performance for arbitrarily many colours has not. In this paper, we present heuristic analysis of the average-case performance of three deterministic algorithms that appear in the literature. We empirically validate our analysis with large scale simulations.
Successor Features for Transfer in Reinforcement Learning
Venue: arXiv cs.AI 2018
Year: 2018

Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks. We propose a transfer framework for the scenario where the reward function changes between tasks but the environment's dynamics remain the same. Our approach rests on two key ideas: "successor features", a value function representation that decouples the dynamics of the environment from the rewards, and "generalized policy improvement", a generalization of dynamic programming's policy improvement operation that considers a set of policies rather than a single one. Put together, the two ideas lead to an approach that integrates seamlessly within the reinforcement learning framework and allows the free exchange of information across tasks. The proposed method also provides performance guarantees for the transferred policy even before any learning has taken place. We derive two theorems that set our approach in firm theoretical ground and present experiments that show that it successfully promotes transfer in practice, significantly outperforming alternative methods in a sequence of navigation tasks and in the control of a simulated robotic arm.
Asymptotically Optimal Approximation Algorithms for Coflow Scheduling
Venue: arXiv cs.DC 2018
Year: 2018

Many modern datacenter applications involve large-scale computations composed of multiple data flows that need to be completed over a shared set of distributed resources. Such a computation completes when all of its flows complete. A useful abstraction for modeling such scenarios is a {\em coflow}, which is a collection of flows (e.g., tasks, packets, data transmissions) that all share the same performance goal. In this paper, we present the first approximation algorithms for scheduling coflows over general network topologies with the objective of minimizing total weighted completion time. We consider two different models for coflows based on the nature of individual flows: circuits, and packets. We design constant-factor polynomial-time approximation algorithms for scheduling packet-based coflows with or without given flow paths, and circuit-based coflows with given flow paths. Furthermore, we give an $O(\log n/\log \log n)$-approximation polynomial time algorithm for scheduling circuit-based coflows where flow paths are not given (here $n$ is the number of network edges). We obtain our results by developing a general framework for coflow schedules, based on interval-indexed linear programs, which may extend to other coflow models and objective functions and may also yield improved approximation bounds for specific network scenarios. We also present an experimental evaluation of our approach for circuit-based coflows that show a performance improvement of at least 22% on average over competing heuristics.
Fast and accurate normalization of vectors and quaternions
Venue: arXiv cs.CG 2018
Year: 2018

We present fast and accurate ways to normalize two and three dimensional vectors and quaternions and compute their length. Our approach is an adaptation of ideas used in the linear algebra library LAPACK, and we believe that the computational geometry and computer aided design communities are not aware of the possibility of speeding up these fundamental operations in the robust way proposed here.
Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs
Venue: arXiv cs.CL 2018
Year: 2018

We present a transition-based parser that jointly produces syntactic and semantic dependencies. It learns a representation of the entire algorithm state, using stack long short-term memories. Our greedy inference algorithm has linear time, including feature extraction. On the CoNLL 2008--9 English shared tasks, we obtain the best published parsing performance among models that jointly learn syntax and semantics.
The parameterized complexity of finding secluded solutions to some classical optimization problems on graphs
Venue: arXiv cs.CC 2018
Year: 2018

This work studies the parameterized complexity of finding secluded solutions to classical combinatorial optimization problems on graphs such as finding minimum s-t separators, feedback vertex sets, dominating sets, maximum independent sets, and vertex deletion problems for hereditary graph properties: Herein, one searches not only to minimize or maximize the size of the solution, but also to minimize the size of its neighborhood. This restriction has applications in secure routing and community detection.
The matching relaxation for a class of generalized set partitioning problems
Venue: arXiv cs.DS 2018
Year: 2018

This paper introduces a discrete relaxation for the class of combinatorial optimization problems which can be described by a set partitioning formulation under packing constraints. We present two combinatorial relaxations based on computing maximum weighted matchings in suitable graphs. Besides providing dual bounds, the relaxations are also used on a variable reduction technique and a matheuristic. We show how that general method can be tailored to sample applications, and also perform a successful computational evaluation with benchmark instances of a problem in maritime logistics.