OpenRaiser/Intern-Atlas huggingface dataset:
Challenges in the Decentralised Web: The Mastodon Case
Venue: arXiv cs.NI 2019
Year: 2019

The Decentralised Web (DW) has recently seen a renewed momentum, with a number of DW platforms like Mastodon, Peer-Tube, and Hubzilla gaining increasing traction. These offer alternatives to traditional social networks like Twitter, YouTube, and Facebook, by enabling the operation of web infrastructure and services without centralised ownership or control. Although their services differ greatly, modern DW platforms mostly rely on two key innovations: first, their open source software allows anybody to setup independent servers ("instances") that people can sign-up to and use within a local community; and second, they build on top of federation protocols so that instances can mesh together, in a peer-to-peer fashion, to offer a globally integrated platform. In this paper, we present a measurement-driven exploration of these two innovations, using a popular DW microblogging platform (Mastodon) as a case study. We focus on identifying key challenges that might disrupt continuing efforts to decentralise the web, and empirically highlight a number of properties that are creating natural pressures towards recentralisation. Finally, our measurements shed light on the behaviour of both administrators (i.e., people setting up instances) and regular users who sign-up to the platforms, also discussing a few techniques that may address some of the issues observed.
Flow: Separating Consensus and Compute
Venue: arXiv cs.DC 2019
Year: 2019

Throughput limitations of existing blockchain architectures are one of the most significant hurdles for their wide-spread adoption. Attempts to address this challenge include layer-2 solutions, such as Bitcoin's Lightning or Ethereum's Plasma network, that move work off the main chain. Another prominent technique is sharding, i.e., breaking the network into many interconnected networks. However, these scaling approaches significantly increase the complexity of the programming model by breaking ACID guarantees increasing the cost and time for application development. In this paper, we describe a novel approach where we split the work traditionally assigned to cryptocurrency miners into two different node roles. Specifically, the selection and ordering of transactions are performed independently from their execution. The focus of this paper is to formalize the split of consensus and computation, and prove that this approach increases throughput without compromising security. In contrast to most existing proposals, our approach achieves scaling via separation of concerns, i.e., better utilization of network resources, rather than sharding. This approach allows established programming paradigms for smart contracts (which generally assume transactional atomicity) to persist without introducing additional complexity. We present simulations on a proof-of-concept network of 32 globally distributed nodes. While the consensus algorithm was identical in all simulations (a 2-step-commit protocol with rotating block proposer), block computation was either included in a consensus nodes' regular operations (conventional architecture) or delegated to specialized execution nodes (separation of concerns). Separation of concerns enables our system to achieve a throughput increase by a factor of 56 compared to conventional architectures without loss of safety or decentralization.
CTRL: A Conditional Transformer Language Model for Controllable Generation
Venue: arXiv cs.CL 2019
Year: 2019

Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text. We release CTRL, a 1.63 billion-parameter conditional transformer language model, trained to condition on control codes that govern style, content, and task-specific behavior. Control codes were derived from structure that naturally co-occurs with raw text, preserving the advantages of unsupervised learning while providing more explicit control over text generation. These codes also allow CTRL to predict which parts of the training data are most likely given a sequence. This provides a potential method for analyzing large amounts of data via model-based source attribution. We have released multiple full-sized, pretrained versions of CTRL at https://github.com/salesforce/ctrl.
A Note on Posterior Probability Estimation for Classifiers
Venue: arXiv cs.LG 2019
Year: 2019

One of the central themes in the classification task is the estimation of class posterior probability at a new point $\bf{x}$. The vast majority of classifiers output a score for $\bf{x}$, which is monotonically related to the posterior probability via an unknown relationship. There are many attempts in the literature to estimate this latter relationship. Here, we provide a way to estimate the posterior probability without resorting to using classification scores. Instead, we vary the prior probabilities of classes in order to derive the ratio of pdf's at point $\bf{x}$, which is directly used to determine class posterior probabilities. We consider here the binary classification problem.
Age of Information Aware Cache Updating with File- and Age-Dependent Update Durations
Venue: arXiv cs.IT 2019
Year: 2019

We consider a system consisting of a library of time-varying files, a server that at all times observes the current version of all files, and a cache that at the beginning stores the current versions of all files but afterwards has to update %fresh versions of these files from the server. Unlike previous works, the update duration is not constant but depends on the file and its Age of Information (AoI), i.e., of the time elapsed since it was last updated. The goal of this work is to design an update policy that minimizes the average AoI of all files with respect to a given popularity distribution. Actually a relaxed problem, close to the original optimization problem, is solved and a practical update policy is derived. The update policy relies on the file popularity and on the functions that characterize the update durations of the files depending on their AoI. Numerical simulations show a significant improvement of this new update policy compared to the so-called square-root policy that is optimal under file-independent and constant update durations.
Analysis of Regression Tree Fitting Algorithms in Learning to Rank
Venue: arXiv cs.IR 2019
Year: 2019

In learning to rank area, industry-level applications have been dominated by gradient boosting framework, which fits a tree using least square error principle. While in classification area, another tree fitting principle, weighted least square error, has been widely used, such as LogitBoost and its variants. However, there is a lack of analysis on the relationship between the two principles in the scenario of learning to rank. We propose a new principle named least objective loss based error that enables us to analyze the issue above as well as several important learning to rank models. We also implement two typical and strong systems and conduct our experiments in two real-world datasets. Experimental results show that our proposed method brings moderate improvements over least square error principle.
Interface Automata for Choreographies
Venue: arXiv cs.FL 2019
Year: 2019

Choreographic approaches to message-passing applications can be regarded as an instance of the model-driven development principles. Choreographies specify interactions among distributed participants coordinating among themselves with message-passing at two levels of abstractions. A global view of the application is specified with a model that abstracts away from asynchrony while a local view of the application specifies the communication pattern of each participant. Noteworthy, the latter view can typically be algorithmically obtained by projection of the global view. A crucial element of this approach is to verify the so-called well-formed conditions on global views so that its projections realise a sound communication protocol. We introduce a novel local model, group interface automata, to represent the local view of choreographies and propose a new method to verify the well-formedness of global choreographies. We rely on a recently proposed semantics of global views formalised in terms of pomsets.
Finite Depth and Width Corrections to the Neural Tangent Kernel
Venue: arXiv cs.LG 2019
Year: 2019

We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation is exponential in the ratio of network depth to width. Thus, even in the limit of infinite overparameterization, the NTK is not deterministic if depth and width simultaneously tend to infinity. Moreover, we prove that for such deep and wide networks, the NTK has a non-trivial evolution during training by showing that the mean of its first SGD update is also exponential in the ratio of network depth to width. This is sharp contrast to the regime where depth is fixed and network width is very large. Our results suggest that, unlike relatively shallow and wide networks, deep and wide ReLU networks are capable of learning data-dependent features even in the so-called lazy training regime.
On educating machines
Venue: arXiv cs.AI 2019
Year: 2019

Machine education is an emerging research field that focuses on the problem which is inverse to machine learning. To date, the literature on educating machines is still in its infancy. A fairly low number of methodology and method papers are scattered throughout various formal and informal publication avenues, mainly because the field is not yet well coalesced (with no well established discussion forums or investigation pathways), but also due to the breadth of its potential ramifications and research directions. In this study we bring together the existing literature and organise the discussion into a small number of research directions (out of many) which are to date sufficiently explored to form a minimal critical mass that can push the machine education concept further towards a standalone research field status.
DL2: A Deep Learning-driven Scheduler for Deep Learning Clusters
Venue: arXiv cs.LG 2019
Year: 2019

More and more companies have deployed machine learning (ML) clusters, where deep learning (DL) models are trained for providing various AI-driven services. Efficient resource scheduling is essential for maximal utilization of expensive DL clusters. Existing cluster schedulers either are agnostic to ML workload characteristics, or use scheduling heuristics based on operators' understanding of particular ML framework and workload, which are less efficient or not general enough. In this paper, we show that DL techniques can be adopted to design a generic and efficient scheduler. DL2 is a DL-driven scheduler for DL clusters, targeting global training job expedition by dynamically resizing resources allocated to jobs. DL2 advocates a joint supervised learning and reinforcement learning approach: a neural network is warmed up via offline supervised learning based on job traces produced by the existing cluster scheduler; then the neural network is plugged into the live DL cluster, fine-tuned by reinforcement learning carried out throughout the training progress of the DL jobs, and used for deciding job resource allocation in an online fashion. By applying past decisions made by the existing cluster scheduler in the preparatory supervised learning phase, our approach enables a smooth transition from existing scheduler, and renders a high-quality scheduler in minimizing average training completion time. We implement DL2 on Kubernetes and enable dynamic resource scaling in DL jobs on MXNet. Extensive evaluation shows that DL2 outperforms fairness scheduler (i.e., DRF) by 44.1% and expert heuristic scheduler (i.e., Optimus) by 17.5% in terms of average job completion time.
Strategic Inference with a Single Private Sample
Venue: arXiv cs.GT 2019
Year: 2019

Motivated by applications in cyber security, we develop a simple game model for describing how a learning agent's private information influences an observing agent's inference process. The model describes a situation in which one of the agents (attacker) is deciding which of two targets to attack, one with a known reward and another with uncertain reward. The attacker receives a single private sample from the uncertain target's distribution and updates its belief of the target quality. The other agent (defender) knows the true rewards, but does not see the sample that the attacker has received. This leads to agents possessing asymmetric information: the attacker is uncertain over the parameter of the distribution, whereas the defender is uncertain about the observed sample. After the attacker updates its belief, both the attacker and the defender play a simultaneous move game based on their respective beliefs. We offer a characterization of the pure strategy equilibria of the game and explain how the players' decisions are influenced by their prior knowledge and the payoffs/costs.
Neural Machine Translation with 4-Bit Precision and Beyond
Venue: arXiv cs.CL 2019
Year: 2019

Neural Machine Translation (NMT) is resource intensive. We design a quantization procedure to compress NMT models better for devices with limited hardware capability. Because most neural network parameters are near zero, we employ logarithmic quantization in lieu of fixed-point quantization. However, we find bias terms are less amenable to log quantization but note they comprise a tiny fraction of the model, so we leave them uncompressed. We also propose to use an error-feedback mechanism during retraining, to preserve the compressed model as a stale gradient. We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation. Models can be compressed up to binary precision, albeit with lower quality. The RNN architecture seems to be more robust to quantization, compared to the Transformer.
BAGH -- Comparative study
Venue: arXiv cs.SE 2019
Year: 2019

Process mining is a new emerging research trend over the last decade which focuses on analyzing the processes using event log and data. The raising integration of information systems for the operation of business processes provides the basis for innovative data analysis approaches. Process mining has the strong relationship between with data mining so that it enables the bond between business intelligence approach and business process management. It focuses on end to end processes and is possible because of the growing availability of event data and new process discovery and conformance checking techniques. Process mining aims to discover, monitor and improve real processes by extracting knowledge from event logs readily available in todays information systems. The discovered process models can be used for a variety of analysis purposes. Many companies have adopted Process aware Information Systems for supporting their business processes in some form. These systems typically have their log events related to the actual business process executions. Proper analysis of Process Aware Information Systems execution logs can yield important knowledge and help organizations improve the quality of their services. This paper reviews and compares various process mining algorithms based on their input parameters, the techniques used and the output generated by them.
Petri Net Machines for Human-Agent Interaction
Venue: arXiv cs.AI 2019
Year: 2019

Smart speakers and robots become ever more prevalent in our daily lives. These agents are able to execute a wide range of tasks and actions and, therefore, need systems to control their execution. Current state-of-the-art such as (deep) reinforcement learning, however, requires vast amounts of data for training which is often hard to come by when interacting with humans. To overcome this issue, most systems still rely on Finite State Machines. We introduce Petri Net Machines which present a formal definition for state machines based on Petri Nets that are able to execute concurrent actions reliably, execute and interleave several plans at the same time, and provide an easy to use modelling language. We show their workings based on the example of Human-Robot Interaction in a shopping mall.
Branch-and-cut and iterated local search for the weighted $k$-traveling repairman problem: an application to the maintenance of speed cameras
Venue: arXiv cs.DS 2019
Year: 2019

Private enterprises and governments around the world use speed cameras to control traffic flow and limit speed excess. Cameras may be exposed to difficult weather conditions and typically require frequent maintenance. When deciding the order in which maintenance should be performed, one has to consider both the traveling times between the cameras and the traffic flow that each camera is supposed to monitor. In this paper, we study the problem of routing a set of technicians to repair cameras by minimizing the total weighted latency, that is, the sum of the weighted waiting times of each camera, where the weight is a parameter proportional to the monitored traffic. The resulting problem, called weighted k-traveling repairman problem (wkTRP), is a generalization of the well-known traveling repairman problem and can be used to model a variety of real-world applications. To solve the wkTRP, we propose an iterated local search heuristic and an exact branch-and-cut algorithm enriched with valid inequalities. The effectiveness of the two methods is proved by extensive computational experiments performed both on instances derived from a real-world case study, as well as on benchmark instances from the literature on the wkTRP and on related problems.
AITuning: Machine Learning-based Tuning Tool for Run-Time Communication Libraries
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we address the problem of tuning communication libraries by using a deep reinforcement learning approach. Reinforcement learning is a machine learning technique incredibly effective in solving game-like situations. In fact, tuning a set of parameters in a communication library in order to get better performance in a parallel application can be expressed as a game: Find the right combination/path that provides the best reward. Even though AITuning has been designed to be utilized with different run-time libraries, we focused this work on applying it to the OpenCoarrays run-time communication library, built on top of MPI-3. This work not only shows the potential of using a reinforcement learning algorithm for tuning communication libraries, but also demonstrates how the MPI Tool Information Interface, introduced by the MPI-3 standard, can be used effectively by run-time libraries to improve the performance without human intervention.
Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data
Venue: arXiv cs.LG 2019
Year: 2019

Nowadays, deep neural networks (DNNs) have become the main instrument for machine learning tasks within a wide range of domains, including vision, NLP, and speech. Meanwhile, in an important case of heterogenous tabular data, the advantage of DNNs over shallow counterparts remains questionable. In particular, there is no sufficient evidence that deep learning machinery allows constructing methods that outperform gradient boosting decision trees (GBDT), which are often the top choice for tabular problems. In this paper, we introduce Neural Oblivious Decision Ensembles (NODE), a new deep learning architecture, designed to work with any tabular data. In a nutshell, the proposed NODE architecture generalizes ensembles of oblivious decision trees, but benefits from both end-to-end gradient-based optimization and the power of multi-layer hierarchical representation learning. With an extensive experimental comparison to the leading GBDT packages on a large number of tabular datasets, we demonstrate the advantage of the proposed NODE architecture, which outperforms the competitors on most of the tasks. We open-source the PyTorch implementation of NODE and believe that it will become a universal framework for machine learning on tabular data.
Sample Complexity of Probabilistic Roadmaps via $ε$-nets
Venue: arXiv cs.DS 2019
Year: 2019

We study fundamental theoretical aspects of probabilistic roadmaps (PRM) in the finite time (non-asymptotic) regime. In particular, we investigate how completeness and optimality guarantees of the approach are influenced by the underlying deterministic sampling distribution ${\mathcal{X}}$ and connection radius ${r>0}$. We develop the notion of ${(δ,ε)}$-completeness of the parameters ${\mathcal{X}, r}$, which indicates that for every motion-planning problem of clearance at least ${δ>0}$, PRM using ${\mathcal{X}, r}$ returns a solution no longer than ${1+ε}$ times the shortest $δ$-clear path. Leveraging the concept of $ε$-nets, we characterize in terms of lower and upper bounds the number of samples needed to guarantee ${(δ,ε)}$-completeness. This is in contrast with previous work which mostly considered the asymptotic regime in which the number of samples tends to infinity. In practice, we propose a sampling distribution inspired by $ε$-nets that achieves nearly the same coverage as grids while using significantly fewer samples.
Biometric Blockchain: A Secure Solution for Intelligent Vehicle Data Sharing
Venue: arXiv cs.CR 2019
Year: 2019

The intelligent vehicle (IV) has become a promising technology that could revolutionize our life in smart cities sooner or later. However, it yet suffers from many security vulnerabilities. Traditional security methods are incapable to secure the IV data sharing against malicious attacks. Blockchain, as expected by both research and industry communities, has emerged as a good solution to address these issues. The major issues in IV data sharing are trust, data accuracy and reliability of data sharing in the communication channel. Blockchain technology, previously working for the cryptocurrency, has recently applied to build trust and reliability in peer-to-peer networks with similar topologies of IV data sharing. In this chapter, we present a new framework, namely biometric blockchain (BBC), for secure IV data sharing. In our new scheme, biometric information is exploited as a cue to record who is responsible in the data sharing activities, while the proposed BBC technology serves as the backbone of the IV data-sharing architecture. Hence, the proposed BBC technology provides a more reliable trust environment between the vehicles while personal identities are traceable in the proposed new scheme.
Linear Size Planar Manhattan Network for Convex Point Sets
Venue: arXiv cs.CG 2019
Year: 2019

Let $G = (V, E)$ be an edge-weighted geometric graph such that every edge is horizontal or vertical. The weight of an edge $uv \in E$ is its length. Let $ W_G (u,v)$ denote the length of a shortest path between a pair of vertices $u$ and $v$ in $G$. The graph $G$ is said to be a Manhattan network for a given point set $ P $ in the plane if $P \subseteq V$ and $\forall p,q \in P$, $ W_G (p,q)=|pq|_1$. In addition to $ P$, graph $G$ may also include a set $T$ of Steiner points in its vertex set $V$. In the Manhattan network problem, the objective is to construct a Manhattan network of small size for a set of $ n $ points. This problem was first considered by Gudmundsson et al.\cite{gudmundsson2007small}. They give a construction of a Manhattan network of size $Θ(n \log n)$ for general point set in the plane. We say a Manhattan network is planar if it can be embedded in the plane without any edge crossings. In this paper, we construct a linear size planar Manhattan network for convex point set in linear time using $\mathcal{ O}(n)$ Steiner points. We also show that, even for convex point set, the construction in Gudmundsson et al. \cite{gudmundsson2007small} needs $Ω(n \log n)$ Steiner points and the network may not be planar.