OpenRaiser/Intern-Atlas huggingface dataset:
Bilu-Linial stability, certified algorithms and the Independent Set problem
Venue: arXiv cs.DS 2021
Year: 2021

We study the Maximum Independent Set (MIS) problem under the notion of stability introduced by Bilu and Linial (2010): a weighted instance of MIS is $γ$-stable if it has a unique optimal solution that remains the unique optimum under multiplicative perturbations of the weights by a factor of at most $γ\geq 1$. The goal then is to efficiently recover the unique optimal solution. In this work, we solve stable instances of MIS on several graphs classes: we solve $\widetilde{O}(Δ/\sqrt{\log Δ})$-stable instances on graphs of maximum degree $Δ$, $(k - 1)$-stable instances on $k$-colorable graphs and $(1 + \varepsilon)$-stable instances on planar graphs. For general graphs, we present a strong lower bound showing that there are no efficient algorithms for $O(n^{\frac{1}{2} - \varepsilon})$-stable instances of MIS, assuming the planted clique conjecture. We also give an algorithm for $(\varepsilon n)$-stable instances. As a by-product of our techniques, we give algorithms and lower bounds for stable instances of Node Multiway Cut. Furthermore, we prove a general result showing that the integrality gap of convex relaxations of several maximization problems reduces dramatically on stable instances. Moreover, we initiate the study of certified algorithms, a notion recently introduced by Makarychev and Makarychev (2018), which is a class of $γ$-approximation algorithms that satisfy one crucial property: the solution returned is optimal for a perturbation of the original instance. We obtain $Δ$-certified algorithms for MIS on graphs of maximum degree $Δ$, and $(1+\varepsilon)$-certified algorithms on planar graphs. Finally, we analyze the algorithm of Berman and Furer (1994) and prove that it is a $\left(\frac{Δ+ 1}{3} + \varepsilon\right)$-certified algorithm for MIS on graphs of maximum degree $Δ$ where all weights are equal to 1.
A Note on Indexing Point Sets for Approximate Bottleneck Distance Queries
Venue: arXiv cs.CG 2021
Year: 2021

The {\em bottleneck distance} is a natural measure of the distance between two finite point sets of equal cardinality, defined as the minimum over all bijections between the point sets of the maximum distance between any pair of points put in correspondence by the bijection. In this work, we consider the problem of building a data structure $\mathbb{D}$ that indexes a collection of $m$ planar point sets (of varying sizes) and supports nearest bottleneck distance queries: given a query point set $Q$ of size $n$, we would like to find the point set(s) $P \in \mathbb{D}$ of size $n$ that are closest in terms of bottleneck distance. Without loss of generality, we assume that all point sets belong to the unit box $[0,1]^2$ in the plane and focus on the $L_\infty$ norm, although the techniques can also be used for other norms. The main contribution is a {\em trie}-based data structure finds a $6$-approximate nearest neighbor in $O(-\lg(d_B(\mathbb{D},Q)) n)$ time, where $d_B(\mathbb{D},Q)$ is the minimum bottleneck distance from $Q$ to any point set in $\mathbb{D}$.
Nonconvex and Nonsmooth Sparse Optimization via Adaptively Iterative Reweighted Methods
Venue: arXiv cs.IT 2021
Year: 2021

We propose a general formulation of nonconvex and nonsmooth sparse optimization problems with convex set constraint, which can take into account most existing types of nonconvex sparsity-inducing terms, bringing strong applicability to a wide range of applications. We design a general algorithmic framework of iteratively reweighted algorithms for solving the proposed nonconvex and nonsmooth sparse optimization problems, which solves a sequence of weighted convex regularization problems with adaptively updated weights. First-order optimality condition is derived and global convergence results are provided under loose assumptions, making our theoretical results a practical tool for analyzing a family of various reweighted algorithms. The effectiveness and efficiency of our proposed formulation and the algorithms are demonstrated in numerical experiments on various sparse optimization problems.
Fréchet Distance Under Translation: Conditional Hardness and an Algorithm via Offline Dynamic Grid Reachability
Venue: arXiv cs.DS 2021
Year: 2021

The discrete Fréchet distance is a popular measure for comparing polygonal curves. An important variant is the discrete Fréchet distance under translation, which enables detection of similar movement patterns in different spatial domains. For polygonal curves of length $n$ in the plane, the fastest known algorithm runs in time $\tilde{\cal O}(n^{5})$ [Ben Avraham, Kaplan, Sharir '15]. This is achieved by constructing an arrangement of disks of size ${\cal O}(n^{4})$, and then traversing its faces while updating reachability in a directed grid graph of size $N := {\cal O}(n^2)$, which can be done in time $\tilde{\cal O}(\sqrt{N})$ per update [Diks, Sankowski '07]. The contribution of this paper is two-fold. First, although it is an open problem to solve dynamic reachability in directed grid graphs faster than $\tilde{\cal O}(\sqrt{N})$, we improve this part of the algorithm: We observe that an offline variant of dynamic $s$-$t$-reachability in directed grid graphs suffices, and we solve this variant in amortized time $\tilde{\cal O}(N^{1/3})$ per update, resulting in an improved running time of $\tilde{\cal O}(n^{4.66...})$ for the discrete Fréchet distance under translation. Second, we provide evidence that constructing the arrangement of size ${\cal O}(n^{4})$ is necessary in the worst case, by proving a conditional lower bound of $n^{4 - o(1)}$ on the running time for the discrete Fréchet distance under translation, assuming the Strong Exponential Time Hypothesis.
Game of Thrones: Fully Distributed Learning for Multi-Player Bandits
Venue: arXiv cs.GT 2021
Year: 2021

We consider an N-player multi-armed bandit game where each player chooses one out of M arms for T turns. Each player has different expected rewards for the arms, and the instantaneous rewards are independent and identically distributed or Markovian. When two or more players choose the same arm, they all receive zero reward. Performance is measured using the expected sum of regrets, compared with an optimal assignment of arms to players that maximizes the sum of expected rewards. We assume that each player only knows her actions and the reward she received each turn. Players cannot observe the actions of other players, and no communication between players is possible. We present a distributed algorithm and prove that it achieves an expected sum of regrets of near-O\left(\log T\right). This is the first algorithm to achieve a near order optimal regret in this fully distributed scenario. All other works have assumed that either all players have the same vector of expected rewards or that communication between players is possible.
On finite-time and fixed-time consensus algorithms for dynamic networks switching among disconnected digraphs
Venue: arXiv cs.SY 2021
Year: 2021

The aim of this paper is to analyze a class of consensus algorithms with finite-time or fixed-time convergence for dynamic networks formed by agents with first-order dynamics. In particular, in the analyzed class a single evaluation of a nonlinear function of the consensus error is performed per each node. The classical assumption of switching among connected graphs is dropped here, allowing to represent failures and intermittent communications between agents. Thus, conditions to guarantee finite and fixed-time convergence, even while switching among disconnected graphs, are provided. Moreover, the algorithms of the considered class are shown to be computationally simpler than previously proposed finite-time consensus algorithms for dynamic networks, which is an important feature in scenarios with computationally limited nodes and energy efficiency requirements such as in sensor networks. The performance of the considered consensus algorithms is illustrated through simulations, comparing it to existing approaches for dynamic networks with finite-time and fixed-time convergence. It is shown that the settling time of the considered algorithms grows slower when the number of nodes increases than with other consensus algorithms for dynamic networks.
Chasing Nested Convex Bodies Nearly Optimally
Venue: arXiv cs.DS 2021
Year: 2021

The convex body chasing problem, introduced by Friedman and Linial, is a competitive analysis problem on any normed vector space. In convex body chasing, for each timestep $t\in\mathbb N$, a convex body $K_t\subseteq \mathbb R^d$ is given as a request, and the player picks a point $x_t\in K_t$. The player aims to ensure that the total distance $\sum_{t=0}^{T-1}||x_t-x_{t+1}||$ is within a bounded ratio of the smallest possible offline solution. In this work, we consider the nested version of the problem, in which the sequence $(K_t)$ must be decreasing. For Euclidean spaces, we consider a memoryless algorithm which moves to the so-called Steiner point, and show that in a certain sense it is exactly optimal among memoryless algorithms. For general finite dimensional normed spaces, we combine the Steiner point and our recent previous algorithm to obtain a new algorithm which is nearly optimal for all $\ell^p_d$ spaces with $p\geq 1$, closing a polynomial gap.
Truly Concurrent Process Algebra to Unifying Quantum and Classical Computing
Venue: arXiv cs.LO 2021
Year: 2021

Truly concurrent process algebras are generalizations to the traditional process algebras for true concurrency, CTC to CCS, APTC to ACP, $π_{tc}$ to $π$ calculus, APPTC to probabilistic process algebra. Now, it is the time to utilize truly concurrent process algebras APTC and APPTC to model quantum computing and unify quantum and classical computing. In this book, we introduce the preliminaries, the utilization of APTC to unify quantum and classical computing and its usage in verification of quantum communication protocols, the utilization of APPTC to unifying quantum and classical computing and its usage in verification of quantum communication protocols.
Robust H-infinity kinematic control of manipulator robots using dual quaternion algebra
Venue: arXiv cs.RO 2021
Year: 2021

This paper proposes a robust dual-quaternion based H-infinity task-space kinematic controller for robot manipulators. To address the manipulator liability to modeling errors, uncertainties, exogenous disturbances, and their influence upon the kinematics of the end-effector pose, we adapt H-infinity techniques\textemdash suitable only for additive noises\textemdash to unit dual quaternions. The noise to error attenuation within the H-infinity framework has the additional advantage of casting aside requirements concerning noise distributions, which are significantly hard to characterize within the group of rigid-body transformations. Using dual quaternion algebra, we provide a connection between performance effects over the end-effector trajectory and different sources of uncertainties and disturbances while satisfying attenuation requirements with minimum instantaneous control effort. The result is an easy-to-implement closed-form H-infinity control design criterion. The performance of the proposed strategy is evaluated within different realistic simulated scenarios and validated through real experiments.
SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks
Venue: arXiv cs.DC 2021
Year: 2021

The performance and efficiency of distributed training of Deep Neural Networks highly depend on the performance of gradient averaging among all participating nodes, which is bounded by the communication between nodes. There are two major strategies to reduce communication overhead: one is to hide communication by overlapping it with computation, and the other is to reduce message sizes. The first solution works well for linear neural architectures, but latest networks such as ResNet and Inception offer limited opportunity for this overlapping. Therefore, researchers have paid more attention to minimizing communication. In this paper, we present a novel gradient compression framework derived from insights of real gradient distributions, and which strikes a balance between compression ratio, accuracy, and computational overhead. Our framework has two major novel components: sparsification of gradients in the frequency domain, and a range-based floating point representation to quantize and further compress gradients frequencies. Both components are dynamic, with tunable parameters that achieve different compression ratio based on the accuracy requirement and systems' platforms, and achieve very high throughput on GPUs. We prove that our techniques guarantee the convergence with a diminishing compression ratio. Our experiments show that the proposed compression framework effectively improves the scalability of most popular neural networks on a 32 GPU cluster to the baseline of no compression, without compromising the accuracy and convergence speed.
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks
Venue: arXiv cs.CV 2021
Year: 2021

We introduce an approach to training a given compact network. To this end, we leverage over-parameterization, which typically improves both neural network optimization and generalization. Specifically, we propose to expand each linear layer of the compact network into multiple consecutive linear layers, without adding any nonlinearity. As such, the resulting expanded network, or ExpandNet, can be contracted back to the compact one algebraically at inference. In particular, we introduce two convolutional expansion strategies and demonstrate their benefits on several tasks, including image classification, object detection, and semantic segmentation. As evidenced by our experiments, our approach outperforms both training the compact network from scratch and performing knowledge distillation from a teacher. Furthermore, our linear over-parameterization empirically reduces gradient confusion during training and improves the network generalization.
Stochastic Top-$K$ Subset Bandits with Linear Space and Non-Linear Feedback
Venue: arXiv cs.LG 2021
Year: 2021

Many real-world problems like Social Influence Maximization face the dilemma of choosing the best $K$ out of $N$ options at a given time instant. This setup can be modeled as a combinatorial bandit which chooses $K$ out of $N$ arms at each time, with an aim to achieve an efficient trade-off between exploration and exploitation. This is the first work for combinatorial bandits where the feedback received can be a non-linear function of the chosen $K$ arms. The direct use of multi-armed bandit requires choosing among $N$-choose-$K$ options making the state space large. In this paper, we present a novel algorithm which is computationally efficient and the storage is linear in $N$. The proposed algorithm is a divide-and-conquer based strategy, that we call CMAB-SM. Further, the proposed algorithm achieves a \textit{regret bound} of $\tilde O(K^{\frac{1}{2}}N^{\frac{1}{3}}T^{\frac{2}{3}})$ for a time horizon $T$, which is \textit{sub-linear} in all parameters $T$, $N$, and $K$. %When applied to the problem of Social Influence Maximization, the performance of the proposed algorithm surpasses the UCB algorithm and some more sophisticated domain-specific methods.
A micro Lie theory for state estimation in robotics
Venue: arXiv cs.RO 2021
Year: 2021

A Lie group is an old mathematical abstract object dating back to the XIX century, when mathematician Sophus Lie laid the foundations of the theory of continuous transformation groups. As it often happens, its usage has spread over diverse areas of science and technology many years later. In robotics, we are recently experiencing an important trend in its usage, at least in the fields of estimation, and particularly in motion estimation for navigation. Yet for a vast majority of roboticians, Lie groups are highly abstract constructions and therefore difficult to understand and to use. This may be due to the fact that most of the literature on Lie theory is written by and for mathematicians and physicists, who might be more used than us to the deep abstractions this theory deals with. In estimation for robotics it is often not necessary to exploit the full capacity of the theory, and therefore an effort of selection of materials is required. In this paper, we will walk through the most basic principles of the Lie theory, with the aim of conveying clear and useful ideas, and leave a significant corpus of the Lie theory behind. Even with this mutilation, the material included here has proven to be extremely useful in modern estimation algorithms for robotics, especially in the fields of SLAM, visual odometry, and the like. Alongside this micro Lie theory, we provide a chapter with a few application examples, and a vast reference of formulas for the major Lie groups used in robotics, including most jacobian matrices and the way to easily manipulate them. We also present a new C++ template-only library implementing all the functionality described here.
Automatic Generation of Dense Non-rigid Optical Flow
Venue: arXiv cs.CV 2021
Year: 2021

There hardly exists any large-scale datasets with dense optical flow of non-rigid motion from real-world imagery as of today. The reason lies mainly in the required setup to derive ground truth optical flows: a series of images with known camera poses along its trajectory, and an accurate 3D model from a textured scene. Human annotation is not only too tedious for large databases, it can simply hardly contribute to accurate optical flow. To circumvent the need for manual annotation, we propose a framework to automatically generate optical flow from real-world videos. The method extracts and matches objects from video frames to compute initial constraints, and applies a deformation over the objects of interest to obtain dense optical flow fields. We propose several ways to augment the optical flow variations. Extensive experimental results show that training on our automatically generated optical flow outperforms methods that are trained on rigid synthetic data using FlowNet-S, LiteFlowNet, PWC-Net, and RAFT. Datasets and implementation of our optical flow generation framework are released at https://github.com/lhoangan/arap_flow
GC-LSTM: Graph Convolution Embedded LSTM for Dynamic Link Prediction
Venue: arXiv cs.SI 2021
Year: 2021

Dynamic link prediction is a research hot in complex networks area, especially for its wide applications in biology, social network, economy and industry. Compared with static link prediction, dynamic one is much more difficult since network structure evolves over time. Currently most researches focus on static link prediction which cannot achieve expected performance in dynamic network. Aiming at low AUC, high Error Rate, add/remove link prediction difficulty, we propose GC-LSTM, a Graph Convolution Network (GC) embedded Long Short Term Memory network (LTSM), for end-to-end dynamic link prediction. To the best of our knowledge, it is the first time that GCN embedded LSTM is put forward for link prediction of dynamic networks. GCN in this new deep model is capable of node structure learning of network snapshot for each time slide, while LSTM is responsible for temporal feature learning for network snapshot. Besides, current dynamic link prediction method can only handle removed links, GC-LSTM can predict both added or removed link at the same time. Extensive experiments are carried out to testify its performance in aspects of prediction accuracy, Error Rate, add/remove link prediction and key link prediction. The results prove that GC-LSTM outperforms current state-of-art method.
TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions
Venue: arXiv cs.RO 2021
Year: 2021

We present a new algorithm for predicting the near-term trajectories of road-agents in dense traffic videos. Our approach is designed for heterogeneous traffic, where the road-agents may correspond to buses, cars, scooters, bicycles, or pedestrians. We model the interactions between different road-agents using a novel LSTM-CNN hybrid network for trajectory prediction. In particular, we take into account heterogeneous interactions that implicitly accounts for the varying shapes, dynamics, and behaviors of different road agents. In addition, we model horizon-based interactions which are used to implicitly model the driving behavior of each road-agent. We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories. We outperform state-of-the-art methods on dense traffic datasets by 30%.
A3: Assisting Android API Migrations Using Code Examples
Venue: arXiv cs.SE 2021
Year: 2021

The fast-paced evolution of Android APIs has posed a challenging task for Android app developers. To leverage Android's frequently released APIs, developers must often spend considerable effort on API migrations. Prior research and Android official documentation typically provide enough information to guide developers in identifying the API calls that must be migrated and the corresponding API calls in an updated version of Android (what to migrate). However, API migration remains a challenging task since developers lack the knowledge of how to migrate the API calls. There exist code examples, such as Google Samples, that illustrate the usage of APIs. We posit that by analyzing the changes of API usage in code examples, we can learn API migration patterns to assist developers with API Migrations. In this paper, we propose an approach that learns API migration patterns from code examples, applies these patterns to the source code of Android apps for API migration, and presents the results to users as potential migration solutions. To evaluate our approach, we migrate API calls in open source Android apps by learning API migration patterns from code examples. We find that our approach can successfully learn API migration patterns and provide API migration assistance in 71 out of 80 cases. Our approach can either migrate API calls with little to no extra modifications needed or provide guidance to assist with the migrations. Through a user study, we find that adopting our approach can reduce the time spent on migrating APIs, on average, by 29%. Moreover, our interviews with app developers highlight the benefits of our approach when seeking API migrations. Our approach demonstrates the value of leveraging the knowledge contained in software repositories to facilitate API migrations.
Attend, Copy, Parse -- End-to-end information extraction from documents
Venue: arXiv cs.CL 2021
Year: 2021

Document information extraction tasks performed by humans create data consisting of a PDF or document image input, and extracted string outputs. This end-to-end data is naturally consumed and produced when performing the task because it is valuable in and of itself. It is naturally available, at no additional cost. Unfortunately, state-of-the-art word classification methods for information extraction cannot use this data, instead requiring word-level labels which are expensive to create and consequently not available for many real life tasks. In this paper we propose the Attend, Copy, Parse architecture, a deep neural network model that can be trained directly on end-to-end data, bypassing the need for word-level labels. We evaluate the proposed architecture on a large diverse set of invoices, and outperform a state-of-the-art production system based on word classification. We believe our proposed architecture can be used on many real life information extraction tasks where word classification cannot be used due to a lack of the required word-level labels.