OpenRaiser/Intern-Atlas huggingface dataset:
Genetic Algorithms and the Traveling Salesman Problem a historical Review
Venue: arXiv cs.NE 2019
Year: 2019

In this paper a highly abstracted view on the historical development of Genetic Algorithms for the Traveling Salesman Problem is given. In a meta-data analysis three phases in the development can be distinguished. First exponential growth in interest till 1996 can be observed, growth stays linear till 2011 and after that publications deteriorate. These three phases are examined and the major milestones are presented. Lastly an outlook to future work in this field is infered.
A Temporal Attentive Approach for Video-Based Pedestrian Attribute Recognition
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we first tackle the problem of pedestrian attribute recognition by video-based approach. The challenge mainly lies in spatial and temporal modeling and how to integrating them for effective and dynamic pedestrian representation. To solve this problem, a novel multi-task model based on the conventional neural network and temporal attention strategy is proposed. Since publicly available dataset is rare, two new large-scale video datasets with expanded attribute definition are presented, on which the effectiveness of both video-based pedestrian attribute recognition methods and the proposed new network architecture is well demonstrated. The two datasets are published on http://irip.buaa.edu.cn/mars_duke_attributes/index.html.
Unsupervised Graph-based Rank Aggregation for Improved Retrieval
Venue: arXiv cs.IR 2019
Year: 2019

This paper presents a robust and comprehensive graph-based rank aggregation approach, used to combine results of isolated ranker models in retrieval tasks. The method follows an unsupervised scheme, which is independent of how the isolated ranks are formulated. Our approach is able to combine arbitrary models, defined in terms of different ranking criteria, such as those based on textual, image or hybrid content representations. We reformulate the ad-hoc retrieval problem as a document retrieval based on fusion graphs, which we propose as a new unified representation model capable of merging multiple ranks and expressing inter-relationships of retrieval results automatically. By doing so, we claim that the retrieval system can benefit from learning the manifold structure of datasets, thus leading to more effective results. Another contribution is that our graph-based aggregation formulation, unlike existing approaches, allows for encapsulating contextual information encoded from multiple ranks, which can be directly used for ranking, without further computations and post-processing steps over the graphs. Based on the graphs, a novel similarity retrieval score is formulated using an efficient computation of minimum common subgraphs. Finally, another benefit over existing approaches is the absence of hyperparameters. A comprehensive experimental evaluation was conducted considering diverse well-known public datasets, composed of textual, image, and multimodal documents. Performed experiments demonstrate that our method reaches top performance, yielding better effectiveness scores than state-of-the-art baseline methods and promoting large gains over the rankers being fused, thus demonstrating the successful capability of the proposal in representing queries based on a unified graph-based model of rank fusions.
The Oracle of DLphi
Venue: arXiv cs.LG 2019
Year: 2019

We present a novel technique based on deep learning and set theory which yields exceptional classification and prediction results. Having access to a sufficiently large amount of labelled training data, our methodology is capable of predicting the labels of the test data almost always even if the training data is entirely unrelated to the test data. In other words, we prove in a specific setting that as long as one has access to enough data points, the quality of the data is irrelevant.
Message-Passing Neural Networks Learn Little's Law
Venue: arXiv cs.NI 2019
Year: 2019

The paper presents a solution to the problem of universal representation of graphs exemplifying communication network topologies with the help of neural networks. The proposed approach is based on message-passing neural networks (MPNN). The approach enables us to represent topologies and operational aspects of networks. The usefulness of the solution is illustrated with a case study of delay prediction in queuing networks. This shows that performance evaluation can be provided without having to apply complex modeling. In consequence, the proposed solution makes it possible to effectively apply methods elaborated in the field of machine learning in communications.
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads
Venue: arXiv cs.DC 2019
Year: 2019

With widespread advances in machine learning, a number of large enterprises are beginning to incorporate machine learning models across a number of products. These models are typically trained on shared, multi-tenant GPU clusters. Similar to existing cluster computing workloads, scheduling frameworks aim to provide features like high efficiency, resource isolation, fair sharing across users, etc. However Deep Neural Network (DNN) based workloads, predominantly trained on GPUs, differ in two significant ways from traditional big data analytics workloads. First, from a cluster utilization perspective, GPUs represent a monolithic resource that cannot be shared at a fine granularity across users. Second, from a workload perspective, deep learning frameworks require gang scheduling reducing the flexibility of scheduling and making the jobs themselves inelastic to failures at runtime. In this paper we present a detailed workload characterization of a two-month long trace from a multi-tenant GPU cluster in a large enterprise. By correlating scheduler logs with logs from individual jobs, we study three distinct issues that affect cluster utilization for DNN training workloads on multi-tenant clusters: (1) the effect of gang scheduling and locality constraints on queuing, (2) the effect of locality on GPU utilization, and (3) failures during training. Based on our experience running a large-scale operation, we provide design guidelines pertaining to next-generation cluster schedulers for DNN training workloads.
Attentive Neural Processes
Venue: arXiv cs.LG 2019
Year: 2019

Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, and can learn a wide family of conditional distributions; they learn predictive distributions conditioned on context sets of arbitrary size. Nonetheless, we show that NPs suffer a fundamental drawback of underfitting, giving inaccurate predictions at the inputs of the observed data they condition on. We address this issue by incorporating attention into NPs, allowing each input location to attend to the relevant context points for the prediction. We show that this greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled.
Multi-hop Data Fragmentation in Unattended Wireless Sensor Networks
Venue: arXiv cs.NI 2019
Year: 2019

In this work, we analyze the advantages of multi-hop data fragmentation in unattended wireless sensor networks (UWSN) and propose a lightweight protocol to achieve it. UWSN has recently become an important aspect in various areas of sensor networks where real-time data collection is difficult to manage. However, the characteristics of UWSN also poses new problems especially in data protection. For more efficient protection, data fragmentation has been proposed to fragment sensing data, which prevents attackers from successfully exploiting the data. However, there are currently minimal work on the strategies of the placement of fragments inside a sensor network. Through this work, we analyze the effects of multi-hop fragment dispersal in relation to effectiveness of data protection and energy consumption. Furthermore, we design a new routing algorithm suitable for the energy-efficient placement of data fragments in UWSN. We utilize simulation-based modeling and testbed implementation via FIT/IoT-Lab to prove the effectiveness of our work.
Supportive Oracles for Parameterized Polynomial-Time Sub-Linear-Space Computations in Relation to L, NL, and P
Venue: arXiv cs.CC 2019
Year: 2019

We focus our attention onto polynomial-time sub-linear-space computation for decision problems, which are parameterized by size parameters $m(x)$, where the informal term "sub linear" means a function of the form $m(x)^{\varepsilon}\cdot polylog(|x|)$ on input instances $x$ for a certain absolute constant $\varepsilon\in(0,1)$ and a certain polylogarithmic function $polylog(n)$. The parameterized complexity class PsubLIN consists of all parameterized decision problems solvable simultaneously in polynomial time using sub-linear space. This complexity class is associated with the linear space hypothesis. There is no known inclusion relationships between PsubLIN and para-NL (nondeterministic log-space class), where the prefix "para-" indicates the natural parameterization of a given complexity class. Toward circumstantial evidences for the inclusions and separations of the associated complexity classes, we seek their relativizations. However, the standard relativization of Turing machines is known to violate the relationships of L$\subseteq$NL=co-NL$\subseteq$DSPACE[O($\log^2{n}$)]$\cap$P. We instead consider special oracles, called NL-supportive oracles, which guarantee these relationships in the corresponding relativized worlds. This paper vigorously constructs such NL-supportive oracles that generate relativized worlds where, for example, para-L$\neq$para-NL$\nsubseteq$PsubLIN and para-L$\neq$para-NL$\subseteq$PsubLIN.
Locality in Index Coding for Large Min-Rank
Venue: arXiv cs.IT 2019
Year: 2019

An index code is said to be locally decodable if each receiver can decode its demand using its side information and by querying only a subset of the transmitted codeword symbols instead of observing the entire codeword. Local decodability can be a beneficial feature in some communication scenarios, such as when the receivers can afford to listen to only a part of the transmissions because of limited availability of power. The locality of an index code is the ratio of the maximum number of codeword symbols queried by a receiver to the message length. In this paper we analyze the optimum locality of linear codes for the family of index coding problems whose min-rank is one less than the number of receivers in the network. We first derive the optimal trade-off between the index coding rate and locality with vector linear coding when the side information graph is a directed cycle. We then provide the optimal trade-off achieved by scalar linear coding for a larger family of problems, viz., problems where the min-rank is only one less than the number of receivers. While the arguments used for achievability are based on known coding techniques, the converse arguments rely on new results on the structure of locally decodable index codes.
Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies
Venue: arXiv cs.SE 2019
Year: 2019

Benchmarks of bugs are essential to empirically evaluate automatic program repair tools. In this paper, we present Bears, a project for collecting and storing bugs into an extensible bug benchmark for automatic repair studies in Java. The collection of bugs relies on commit building state from Continuous Integration (CI) to find potential pairs of buggy and patched program versions from open-source projects hosted on GitHub. Each pair of program versions passes through a pipeline where an attempt of reproducing a bug and its patch is performed. The core step of the reproduction pipeline is the execution of the test suite of the program on both program versions. If a test failure is found in the buggy program version candidate and no test failure is found in its patched program version candidate, a bug and its patch were successfully reproduced. The uniqueness of Bears is the usage of CI (builds) to identify buggy and patched program version candidates, which has been widely adopted in the last years in open-source projects. This approach allows us to collect bugs from a diversity of projects beyond mature projects that use bug tracking systems. Moreover, Bears was designed to be publicly available and to be easily extensible by the research community through automatic creation of branches with bugs in a given GitHub repository, which can be used for pull requests in the Bears repository. We present in this paper the approach employed by Bears, and we deliver the version 1.0 of Bears, which contains 251 reproducible bugs collected from 72 projects that use the Travis CI and Maven build environment.
Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent
Venue: arXiv cs.LG 2019
Year: 2019

We interpret the variational inference of the Stochastic Gradient Descent (SGD) as minimizing a new potential function named the \textit{quasi-potential}. We analytically construct the quasi-potential function in the case when the loss function is convex and admits only one global minimum point. We show in this case that the quasi-potential function is related to the noise covariance structure of SGD via a partial differential equation of Hamilton-Jacobi type. This relation helps us to show that anisotropic noise leads to faster escape than isotropic noise. We then consider the dynamics of SGD in the case when the loss function is non-convex and admits several different local minima. In this case, we demonstrate an example that shows how the noise covariance structure plays a role in "implicit regularization", a phenomenon in which SGD favors some particular local minimum points. This is done through the relation between the noise covariance structure and the quasi-potential function. Our analysis is based on Large Deviations Theory (LDT), and they are validated by numerical experiments.
WALL-E: An Efficient Reinforcement Learning Research Framework
Venue: arXiv cs.LG 2019
Year: 2019

There are two halves to RL systems: experience collection time and policy learning time. For a large number of samples in rollouts, experience collection time is the major bottleneck. Thus, it is necessary to speed up the rollout generation time with multi-process architecture support. Our work, dubbed WALL-E, utilizes multiple rollout samplers running in parallel to rapidly generate experience. Due to our parallel samplers, we experience not only faster convergence times, but also higher average reward thresholds. For example, on the MuJoCo HalfCheetah-v2 task, with $N = 10$ parallel sampler processes, we are able to achieve much higher average return than those from using only a single process architecture.
Transfer Learning and Meta Classification Based Deep Churn Prediction System for Telecom Industry
Venue: arXiv cs.LG 2019
Year: 2019

A churn prediction system guides telecom service providers to reduce revenue loss. However, the development of a churn prediction system for a telecom industry is a challenging task, mainly due to the large size of the data, high dimensional features, and imbalanced distribution of the data. In this paper, we present a solution to the inherent problems of churn prediction, using the concept of Transfer Learning (TL) and Ensemble-based Meta-Classification. The proposed method TL-DeepE is applied in two stages. The first stage employs TL by fine-tuning multiple pre-trained Deep Convolution Neural Networks (CNNs). Telecom datasets are normally in vector form, which is converted into 2D images because Deep CNNs have high learning capacity on images. In the second stage, predictions from these Deep CNNs are appended to the original feature vector and thus are used to build a final feature vector for the high-level Genetic Programming (GP) and AdaBoost based ensemble classifier. Thus, the experiments are conducted using various CNNs as base classifiers and the GP-AdaBoost as a meta-classifier. By using 10-fold cross-validation, the performance of the proposed TL-DeepE system is compared with existing techniques, for two standard telecommunication datasets; Orange and Cell2cell. Performing experiments on Orange and Cell2cell datasets, the prediction accuracy obtained was 75.4% and 68.2%, while the area under the curve was 0.83 and 0.74, respectively.
Taming Distrust in the Decentralized Internet with PIXIU
Venue: arXiv cs.CR 2019
Year: 2019

Decentralized Internet is booming. People are fascinated by its promise that users can truly own their data. However, in a decentralized Internet, completing a task usually involves multiple nodes with mutual distrust. Such distrust might eventually become a major obstacle for the growth of the decentralized Internet. In this paper, we analyze the distrust using a simple model and highlight the properties required to faithfully accomplish one task in a decentralized Internet. We also introduce our draft solution -- PIXIU, a framework to mitigate the distrust among different nodes. In PIXIU, we design and utilize trust-λ and decentralized executor to achieve the above-needed properties.
Linearized ADMM and Fast Nonlocal Denoising for Efficient Plug-and-Play Restoration
Venue: arXiv cs.CV 2019
Year: 2019

In plug-and-play image restoration, the regularization is performed using powerful denoisers such as nonlocal means (NLM) or BM3D. This is done within the framework of alternating direction method of multipliers (ADMM), where the regularization step is formally replaced by an off-the-shelf denoiser. Each plug-and-play iteration involves the inversion of the forward model followed by a denoising step. In this paper, we present a couple of ideas for improving the efficiency of the inversion and denoising steps. First, we propose to use linearized ADMM, which generally allows us to perform the inversion at a lower cost than standard ADMM. Moreover, we can easily incorporate hard constraints into the optimization framework as a result. Second, we develop a fast algorithm for doubly stochastic NLM, originally proposed by Sreehari et al. (IEEE TCI, 2016), which is about 80x faster than brute-force computation. This particular denoiser can be expressed as the proximal map of a convex regularizer and, as a consequence, we can guarantee convergence for linearized plug-and-play ADMM. We demonstrate the effectiveness of our proposals for super-resolution and single-photon imaging.
CRDN: Cascaded Residual Dense Networks for Dynamic MR Imaging with Edge-enhanced Loss Constraint
Venue: arXiv cs.CV 2019
Year: 2019

Dynamic magnetic resonance (MR) imaging has generated great research interest, as it can provide both spatial and temporal information for clinical diagnosis. However, slow imaging speed or long scanning time is still one of the challenges for dynamic MR imaging. Most existing methods reconstruct Dynamic MR images from incomplete k-space data under the guidance of compressed sensing (CS) or low rank theory, which suffer from long iterative reconstruction time. Recently, deep learning has shown great potential in accelerating dynamic MR. Our previous work proposed a dynamic MR imaging method with both k-space and spatial prior knowledge integrated via multi-supervised network training. Nevertheless, there was still a certain degree of smooth in the reconstructed images at high acceleration factors. In this work, we propose cascaded residual dense networks for dynamic MR imaging with edge-enhance loss constraint, dubbed as CRDN. Specifically, the cascaded residual dense networks fully exploit the hierarchical features from all the convolutional layers with both local and global feature fusion. We further utilize the total variation (TV) loss function, which has the edge enhancement properties, for training the networks.
Cocktail BPSK: Achievable Data Rate beyond Channel Capacity
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we propose a method layering two independent BPSK symbol streams in parallel transmission through a single additive white Gaussian noise (AWGN) channel and investigate its achievable data rate (ADR). The receiver processes the two layered signals individually, where the critical point is to translate the interference between the two layers into the problem of one BPSK symbol having two equiprobable amplitudes. After detecting one symbol stream, we subtract the detected result to demodulate the other symbol stream. Based on the concept of mutual information, we formulate the ADRs and unearth an interesting phenomenon -- the sum ADR of the two BPSK symbol streams is higher than the channel capacity at low signal to noise ratio (SNR). Our theoretical derivations together with illustrative numerical results substantiate this phenomenon. We refer to this approach as cocktail BPSK because the amplitudes of the two layered BPSK streams can be adjusted at the transmitter to achieve high spectral efficiency.
Cold-start Playlist Recommendation with Multitask Learning
Venue: arXiv cs.IR 2019
Year: 2019

Playlist recommendation involves producing a set of songs that a user might enjoy. We investigate this problem in three cold-start scenarios: (i) cold playlists, where we recommend songs to form new personalised playlists for an existing user; (ii) cold users, where we recommend songs to form new playlists for a new user; and (iii) cold songs, where we recommend newly released songs to extend users' existing playlists. We propose a flexible multitask learning method to deal with all three settings. The method learns from user-curated playlists, and encourages songs in a playlist to be ranked higher than those that are not by minimising a bipartite ranking loss. Inspired by an equivalence between bipartite ranking and binary classification, we show how one can efficiently approximate an optimal solution of the multitask learning objective by minimising a classification loss. Empirical results on two real playlist datasets show the proposed approach has good performance for cold-start playlist recommendation.
Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-Identification
Venue: arXiv cs.CV 2019
Year: 2019

In person re-identification (ReID) task, because of its shortage of trainable dataset, it is common to utilize fine-tuning method using a classification network pre-trained on a large dataset. However, it is relatively difficult to sufficiently fine-tune the low-level layers of the network due to the gradient vanishing problem. In this work, we propose a novel fine-tuning strategy that allows low-level layers to be sufficiently trained by rolling back the weights of high-level layers to their initial pre-trained weights. Our strategy alleviates the problem of gradient vanishing in low-level layers and robustly trains the low-level layers to fit the ReID dataset, thereby increasing the performance of ReID tasks. The improved performance of the proposed strategy is validated via several experiments. Furthermore, without any add-ons such as pose estimation or segmentation, our strategy exhibits state-of-the-art performance using only vanilla deep convolutional neural network architecture.