OpenRaiser/Intern-Atlas huggingface dataset:
Stable Opponent Shaping in Differentiable Games
Venue: arXiv cs.MA 2021
Year: 2021

A growing number of learning methods are actually differentiable games whose players optimise multiple, interdependent objectives in parallel -- from GANs and intrinsic curiosity to multi-agent RL. Opponent shaping is a powerful approach to improve learning dynamics in these games, accounting for player influence on others' updates. Learning with Opponent-Learning Awareness (LOLA) is a recent algorithm that exploits this response and leads to cooperation in settings like the Iterated Prisoner's Dilemma. Although experimentally successful, we show that LOLA agents can exhibit 'arrogant' behaviour directly at odds with convergence. In fact, remarkably few algorithms have theoretical guarantees applying across all (n-player, non-convex) games. In this paper we present Stable Opponent Shaping (SOS), a new method that interpolates between LOLA and a stable variant named LookAhead. We prove that LookAhead converges locally to equilibria and avoids strict saddles in all differentiable games. SOS inherits these essential guarantees, while also shaping the learning of opponents and consistently either matching or outperforming LOLA experimentally.
Topology-Aware Surface Reconstruction for Point Clouds
Venue: arXiv cs.CG 2021
Year: 2021

We present an approach to inform the reconstruction of a surface from a point scan through topological priors. The reconstruction is based on basis functions which are optimized to provide a good fit to the point scan while satisfying predefined topological constraints. We optimize the parameters of a model to obtain likelihood function over the reconstruction domain. The topological constraints are captured by persistence diagrams which are incorporated in the optimization algorithm promote the correct topology. The result is a novel topology-aware technique which can: 1.) weed out topological noise from point scans, and 2.) capture certain nuanced properties of the underlying shape which could otherwise be lost while performing surface reconstruction. We showcase results reconstructing shapes with multiple potential topologies, compare to other classical surface construction techniques, and show the completion of real scan data.
Cooperative Multi-Agent Policy Gradients with Sub-optimal Demonstration
Venue: arXiv cs.AI 2021
Year: 2021

Many reality tasks such as robot coordination can be naturally modelled as multi-agent cooperative system where the rewards are sparse. This paper focuses on learning decentralized policies for such tasks using sub-optimal demonstration. To learn the multi-agent cooperation effectively and tackle the sub-optimality of demonstration, a self-improving learning method is proposed: On the one hand, the centralized state-action values are initialized by the demonstration and updated by the learned decentralized policy to improve the sub-optimality. On the other hand, the Nash Equilibrium are found by the current state-action value and are used as a guide to learn the policy. The proposed method is evaluated on the combat RTS games which requires a high level of multi-agent cooperation. Extensive experimental results on various combat scenarios demonstrate that the proposed method can learn multi-agent cooperation effectively. It significantly outperforms many state-of-the-art demonstration based approaches.
Sculptures in Concurrency
Venue: arXiv cs.LO 2021
Year: 2021

We give a formalization of Pratt's intuitive sculpting process for higher-dimensional automata (HDA). Intuitively, an HDA is a sculpture if it can be embedded in (i.e., sculpted from) a single higher dimensional cell (hypercube). A first important result of this paper is that not all HDA can be sculpted, exemplified through several natural acyclic HDA, one being the famous "broken box" example of van Glabbeek. Moreover, we show that even the natural operation of unfolding is completely unrelated to sculpting, e.g., there are sculptures whose unfoldings cannot be sculpted. We investigate the expressiveness of sculptures, as a proper subclass of HDA, by showing them to be equivalent to regular ST-structures (an event-based counterpart of HDA) and to (regular) Chu spaces over 3 (in their concurrent interpretation given by Pratt). We believe that our results shed new light on the intuitions behind sculpting as a method of modeling concurrent behavior, showing the precise reaches of its expressiveness. Besides expressiveness, we also develop an algorithm to decide whether an HDA can be sculpted. More importantly, we show that sculptures are equivalent to Euclidean cubical complexes (being the geometrical counterpart of our combinatorial definition), which include the popular PV models used for deadlock detection. This exposes a close connection between geometric and combinatorial models for concurrency which may be of use for both areas.
An Unpaired Shape Transforming Method for Image Translation and Cross-Domain Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

We address the problem of unpaired geometric image-to-image translation. Rather than transferring the style of an image as a whole, our goal is to translate the geometry of an object as depicted in different domains while preserving its appearance characteristics. Our model is trained in an unpaired fashion, i.e. without the need of paired images during training. It performs all steps of the shape transfer within a single model and without additional post-processing stages. Extensive experiments on the VITON, CMU-Multi-PIE and our own FashionStyle datasets show the effectiveness of the method. In addition, we show that despite their low-dimensionality, the features learned by our model are useful to the item retrieval task.
Weighted Global Normalization for Multiple Choice Reading Comprehension over Long Documents
Venue: arXiv cs.CL 2021
Year: 2021

Motivated by recent evidence pointing out the fragility of high-performing span prediction models, we direct our attention to multiple choice reading comprehension. In particular, this work introduces a novel method for improving answer selection on long documents through weighted global normalization of predictions over portions of the documents. We show that applying our method to a span prediction model adapted for answer selection helps model performance on long summaries from NarrativeQA, a challenging reading comprehension dataset with an answer selection task, and we strongly improve on the task baseline performance by +36.2 Mean Reciprocal Rank.
On the Generalized Degrees of Freedom of Noncoherent Interference Channel
Venue: arXiv cs.IT 2021
Year: 2021

We study the generalized degrees of freedom (gDoF) of the block-fading noncoherent 2-user interference channel (IC) with a coherence time of T symbol durations and symmetric fading statistics. We demonstrate that a natural training-based scheme for the noncoherent IC, is suboptimal in several regimes. We study and analyze several alternate schemes: the first is a new noncoherent scheme using rate-splitting. We also consider a scheme that treats interference-as-noise (TIN) and a time division multiplexing (TDM) scheme. We show that a standard training-based scheme for the noncoherent IC is outperformed by one of these schemes in several regimes: our results demonstrate that in the very weak interference regime, the TIN scheme is the best; in the strong interference regime, the TDM scheme and the noncoherent rate-splitting scheme give better performance; in other cases either of the TIN, TDM or noncoherent rate-splitting scheme could be preferred. We also study the noncoherent IC with feedback and propose another noncoherent rate-splitting scheme. Again for the feedback case, our results demonstrate that a natural training-based scheme can be outperformed by other schemes.
On the Enumeration Complexity of Unions of Conjunctive Queries
Venue: arXiv cs.DB 2021
Year: 2021

We study the enumeration complexity of Unions of Conjunctive Queries(UCQs). We aim to identify the UCQs that are tractable in the sense that the answer tuples can be enumerated with a linear preprocessing phase and a constant delay between every successive tuples. It has been established that, in the absence of self-joins and under conventional complexity assumptions, the CQs that admit such an evaluation are precisely the free-connex ones. A union of tractable CQs is always tractable. We generalize the notion of free-connexity from CQs to UCQs, thus showing that some unions containing intractable CQs are, in fact, tractable. Interestingly, some unions consisting of only intractable CQs are tractable too. We show how to use the techniques presented in this article also in settings where the database contains cardinality dependencies (including functional dependencies and key constraints) or when the UCQs contain disequalities. The question of finding a full characterization of the tractability of UCQs remains open. Nevertheless, we prove that for several classes of queries, free-connexity fully captures the tractable UCQs.
The Crossing Tverberg Theorem
Venue: arXiv cs.CG 2021
Year: 2021

Tverberg's theorem is one of the cornerstones of discrete geometry. It states that, given a set $X$ of at least $(d+1)(r-1)+1$ points in $\mathbb R^d$, one can find a partition $X=X_1\cup \ldots \cup X_r$ of $X$, such that the convex hulls of the $X_i$, $i=1,\ldots,r$, all share a common point. In this paper, we prove a strengthening of this theorem that guarantees a partition which, in addition to the above, has the property that the boundaries of full-dimensional convex hulls have pairwise nonempty intersections. Possible generalizations and algorithmic aspects are also discussed. As a concrete application, we show that any $n$ points in the plane in general position span $\lfloor n/3\rfloor$ vertex-disjoint triangles that are pairwise crossing, meaning that their boundaries have pairwise nonempty intersections; this number is clearly best possible. A previous result of Rebollar et al.\ guarantees $\lfloor n/6\rfloor$ pairwise crossing triangles. Our result generalizes to a result about simplices in $\mathbb R^d,d\ge2$.
Optimizing Data Intensive Flows for Networks on Chips
Venue: arXiv cs.DC 2021
Year: 2021

Data flow analysis and optimization is considered for homogeneous rectangular mesh networks. We propose a flow matrix equation which allows a closed-form characterization of the nature of the minimal time solution, speedup and a simple method to determine when and how much load to distribute to processors. We also propose a rigorous mathematical proof about the flow matrix optimal solution existence and that the solution is unique. The methodology introduced here is applicable to many interconnection networks and switching protocols (as an example we examine toroidal networks and hypercube networks in this paper). An important application is improving chip area and chip scalability for networks on chips processing divisible style loads.
Quantum Equivalence of the DLP and CDHP for Group Actions
Venue: arXiv cs.CR 2021
Year: 2021

In this short note we give a polynomial-time quantum reduction from the vectorization problem (DLP) to the parallelization problem (CDHP) for group actions. Combined with the trivial reduction from par-allelization to vectorization, we thus prove the quantum equivalence of both problems.
Population-Guided Large Margin Classifier for High-Dimension Low -Sample-Size Problems
Venue: arXiv cs.LG 2021
Year: 2021

Various applications in different fields, such as gene expression analysis or computer vision, suffer from data sets with high-dimensional low-sample-size (HDLSS), which has posed significant challenges for standard statistical and modern machine learning methods. In this paper, we propose a novel linear binary classifier, denoted by population-guided large margin classifier (PGLMC), which is applicable to any sorts of data, including HDLSS. PGLMC is conceived with a projecting direction w given by the comprehensive consideration of local structural information of the hyperplane and the statistics of the training samples. Our proposed model has several advantages compared to those widely used approaches. First, it is not sensitive to the intercept term b. Second, it operates well with imbalanced data. Third, it is relatively simple to be implemented based on Quadratic Programming. Fourth, it is robust to the model specification for various real applications. The theoretical properties of PGLMC are proven. We conduct a series of evaluations on two simulated and six real-world benchmark data sets, including DNA classification, digit recognition, medical image analysis, and face recognition. PGLMC outperforms the state-of-the-art classification methods in most cases, or at least obtains comparable results.
Measuring the Correlation of Personal Identity Documents in Structured Format
Venue: arXiv cs.CR 2021
Year: 2021

Personal identity documents play a major role in every citizen's life and the authorities responsible for validating them typically require human intervention to manually cross-check multiple documents belonging to an individual. The world is rapidly replacing physical documents with digital documents where every piece of data is stored digitally in a machine-readable and structured format. In this paper, we describe a technique to extract identity data from a structured data format and calculate a normalized correlation score for personal identity documents. Experimental results show that the proposed technique effectively calculates the correlation score for personal identity documents.
Separation for dot-depth two
Venue: arXiv cs.FL 2021
Year: 2021

The dot-depth hierarchy of Brzozowski and Cohen classifies the star-free languages of finite words. By a theorem of McNaughton and Papert, these are also the first-order definable languages. The dot-depth rose to prominence following the work of Thomas, who proved an exact correspondence with the quantifier alternation hierarchy of first-order logic: each level in the dot-depth hierarchy consists of all languages that can be defined with a prescribed number of quantifier blocks. One of the most famous open problems in automata theory is to settle whether the membership problem is decidable for each level: is it possible to decide whether an input regular language belongs to this level? Despite a significant research effort, membership by itself has only been solved for low levels. A recent breakthrough was achieved by replacing membership with a more general problem: separation. Given two input languages, one has to decide whether there exists a third language in the investigated level containing the first language and disjoint from the second. The motivation is that: (1) while more difficult, separation is more rewarding (2) it provides a more convenient framework (3) all recent membership algorithms are reductions to separation for lower levels. We present a separation algorithm for dot-depth two. While this is our most prominent application, our result is more general. We consider a family of hierarchies that includes the dot-depth: concatenation hierarchies. They are built via a generic construction process. One first chooses an initial class, the basis, which is the lowest level in the hierarchy. Further levels are built by applying generic operations. Our main theorem states that for any concatenation hierarchy whose basis is finite, separation is decidable for level one. In the special case of the dot-depth, this can be lifted to level two using previously known results.
Ubiquitous Acoustic Sensing on Commodity IoT Devices: A Survey
Venue: arXiv cs.SD 2021
Year: 2021

With the proliferation of Internet-of-Things devices, acoustic sensing attracts much attention in recent years. It exploits acoustic transceivers such as microphones and speakers beyond their primary functions, namely recording and playing, to enable novel applications and new user experiences. In this paper, we present the first systematic survey of recent advances in active acoustic sensing using commodity hardware with a frequency range below 24~\!kHz. We propose a general framework that categorizes main building blocks of acoustic sensing systems. This framework encompasses three layers, i.e., physical layer, core technique layer, and application layer. The physical layer includes basic hardware components, acoustic platforms as well as the air-borne and structure-borne channel characteristics. The core technique layer encompasses key mechanisms to generate acoustic signals (waveforms) and to extract useful temporal, spatial and spectral information from received signals. The application layer builds upon the functions offered by the core techniques to realize different acoustic sensing applications. We highlight unique challenges due to the limitations of physical devices and acoustic channels and how they are mitigated or overcame by core processing techniques and application-specific solutions. Finally, research opportunities and future directions are discussed to spawn further in-depth investigation on acoustic sensing.
An Exponential Lower Bound on the Sub-Packetization of MSR Codes
Venue: arXiv cs.IT 2021
Year: 2021

An $(n,k,\ell)$-vector MDS code is a $\mathbb{F}$-linear subspace of $(\mathbb{F}^\ell)^n$ (for some field $\mathbb{F}$) of dimension $k\ell$, such that any $k$ (vector) symbols of the codeword suffice to determine the remaining $r=n-k$ (vector) symbols. The length $\ell$ of each codeword symbol is called the sub-packetization of the code. Such a code is called minimum storage regenerating (MSR), if any single symbol of a codeword can be recovered by downloading $\ell/r$ field elements (which is known to be the least possible) from each of the other symbols. MSR codes are attractive for use in distributed storage systems, and by now a variety of ingenious constructions of MSR codes are available. However, they all suffer from exponentially large sub-packetization $\ell \gtrsim r^{k/r}$. Our main result is an almost tight lower bound showing that for an MSR code, one must have $\ell \ge \exp(Ω(k/r))$. This settles a central open question concerning MSR codes that has received much attention. Previously, a lower bound of $\approx \exp(\sqrt{k/r})$, and a tight lower bound for a restricted class of "optimal access" MSR codes, were known.
Machine learning with neural networks
Venue: arXiv cs.LG 2021
Year: 2021

These are lecture notes for a course on machine learning with neural networks for scientists and engineers that I have given at Gothenburg University and Chalmers Technical University in Gothenburg, Sweden. The material is organised into three parts: Hopfield networks, supervised learning of labeled data, and learning algorithms for unlabeled data sets. Part I introduces stochastic recurrent networks: Hopfield networks and Boltzmann machines. The analysis of their learning rules sets the scene for the later parts. Part II describes supervised learning with multilayer perceptrons and convolutional neural networks. This part starts with a simple geometrical interpretation of the learning rule and leads to the recent successes of convolutional networks in object recognition, recurrent networks in language processing, and reservoir computers in time-series analysis. Part III explains what neural networks can learn about data that is not labeled. This part begins with a description of unsupervised learning techniques for clustering of data, non-linear projections, and embeddings. A section on autoencoders explains how to learn without labels using convolutional networks, and the last chapter is dedicated to reinforcement learning. The overall goal of the course is to explain the fundamental principles that allow neural networks to learn, emphasising ideas and concepts that are common to all three parts. The present version does not contain exercises (copyright owned by Cambridge University Press). The complete book is available at https://www.cambridge.org/gb/academic/subjects/physics/statistical-physics/machine-learning-neural-networks-introduction-scientists-and-engineers?format=HB.
SPARCs for Unsourced Random Access
Venue: arXiv cs.IT 2021
Year: 2021

Unsourced random-access (U-RA) is a type of grant-free random access with a virtually unlimited number of users, of which only a certain number $K_a$ are active on the same time slot. Users employ exactly the same codebook, and the task of the receiver is to decode the list of transmitted messages. We present a concatenated coding construction for U-RA on the AWGN channel, in which a sparse regression code (SPARC) is used as an inner code to create an effective outer OR-channel. Then an outer code is used to resolve the multiple-access interference in the OR-MAC. We propose a modified version of the approximate message passing (AMP) algorithm as an inner decoder and give a precise asymptotic analysis of the error probabilities of the AMP decoder and of a hypothetical optimal inner MAP decoder. This analysis shows that the concatenated construction can achieve a vanishing per-user error probability in the limit of large blocklength and a large number of active users at sum-rates up to the symmetric Shannon capacity, i.e. as long as $K_aR < 0.5\log_2(1+K_a\SNR)$. This extends previous point-to-point optimality results about SPARCs to the unsourced multiuser scenario. Furthermore, we give an optimization algorithm to find the power allocation for the inner SPARC code that minimizes the required $\SNR$.
Cooperative coevolution of real predator robots and virtual robots in the pursuit domain
Venue: arXiv cs.RO 2021
Year: 2021

The pursuit domain, or predator-prey problem is a standard testbed for the study of coordination techniques. In spite that its problem setup is apparently simple, it is challenging for the research of the emerged swarm intelligence. This paper presents a particle swarm optimization (PSO) based cooperative coevolutionary algorithm for the (predator) robots, called CCPSO-R, where real and virtual robots coexist in an evolutionary algorithm (EA). Virtual robots sample and explore the vicinity of the corresponding real robots and act as their action spaces, while the real robots consist of the real predators who actually pursue the prey robot without fixed behavior rules under the immediate guidance of the fitness function, which is designed in a modular manner with very limited domain knowledge. In addition, kinematic limits and collision avoidance considerations are integrated into the update rules of robots. Experiments are conducted on a scalable swarm of predator robots with 4 types of preys, the results of which show the reliability, generality, and scalability of the proposed CCPSO-R. Comparison with a representative dynamic path planning based algorithm Multi-Agent Real-Time Pursuit (MAPS) further shows the effectiveness of CCPSO-R. Finally, the codes of this paper are public available at: https://github.com/LijunSun90/pursuitCCPSOR.