OpenRaiser/Intern-Atlas huggingface dataset:
Limits of Transmit and Receive Array Gain in Massive MIMO
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we consider the transmit and receive antenna array gain of massive MIMO systems. In particular, we look at their dependence on the number of antennas in the array, and the antenna spacing for uniform linear and uniform circular arrays. It is known that the transmit array gain saturates at a certain antenna spacing, but the receive array gain had not been considered. With our physically consistent analysis based on the Multiport Communication Theory, we show that the receive array gain does not saturate, but that there is a peak at a certain antenna spacing when there is no decoupling network at the receiver. As implementing a decoupling network for massive MIMO would be almost impossible, this is a reasonable assumption. Furthermore, we analyze how the array gain changes depending on the antenna spacing and the size of the antenna array and derive design recommendations.
Diffusion and Consensus in a Weakly Coupled Network of Networks
Venue: arXiv cs.SI 2020
Year: 2020

We study diffusion and consensus dynamics in a Network of Networks model. In this model, there is a collection of sub-networks, connected to one another using a small number of links. We consider a setting where the links between networks have small weights, or are used less frequently than links within each sub-network. Using spectral perturbation theory, we analyze the diffusion rate and convergence rate of the investigated systems. Our analysis shows that the first order approximation of the diffusion and convergence rates is independent of the topologies of the individual graphs; the rates depend only on the number of nodes in each graph and the topology of the connecting edges. The second order analysis shows a relationship between the diffusion and convergence rates and the information centrality of the connecting nodes within each sub-network. We further highlight these theoretical results through numerical examples.
RL agents Implicitly Learning Human Preferences
Venue: arXiv cs.AI 2020
Year: 2020

In the real world, RL agents should be rewarded for fulfilling human preferences. We show that RL agents implicitly learn the preferences of humans in their environment. Training a classifier to predict if a simulated human's preferences are fulfilled based on the activations of a RL agent's neural network gets .93 AUC. Training a classifier on the raw environment state gets only .8 AUC. Training the classifier off of the RL agent's activations also does much better than training off of activations from an autoencoder. The human preference classifier can be used as the reward function of an RL agent to make RL agent more beneficial for humans.
Functional Programming in Pattern-Match-Oriented Programming Style
Venue: arXiv cs.PL 2020
Year: 2020

Throughout the history of functional programming, recursion has emerged as a natural method for describing loops in programs. However, there does often exist a substantial cognitive distance between the recursive definition and the simplest explanation of an algorithm even for the basic list processing functions such as map, concat, or unique; when we explain these functions, we seldom use recursion explicitly as we do in functional programming. For example, map is often explained as follows: the map function takes a function and a list and returns a list of the results of applying the function to all the elements of the list. This paper advocates a new programming paradigm called pattern-match-oriented programming for filling this gap. An essential ingredient of our method is utilizing pattern matching for non-free data types. Pattern matching for non-free data types features non-linear pattern matching with backtracking and extensibility of pattern-matching algorithms. Several non-standard pattern constructs, such as not-patterns, loop patterns, and sequential patterns, are derived from this pattern-matching facility. Based on that result, this paper introduces many programming techniques that replace explicit recursions with an intuitive pattern by confining recursions inside patterns. We classify these techniques as pattern-match-oriented programming design patterns. These programming techniques allow us to redefine not only the most basic functions for list processing such as map, concat, or unique more elegantly than the traditional functional programming style, but also more practical mathematical algorithms and software such as a SAT solver, computer algebra system, and database query language that we had not been able to implement concisely.
Implementing a Language for Distributed Systems: Choices and Experiences with Type Level and Macro Programming in Scala
Venue: arXiv cs.PL 2020
Year: 2020

Multitier programming languages reduce the complexity of developing distributed systems by developing the distributed system in a single coherent code base. The compiler or the runtime separate the code for the components of the distributed system, enabling abstraction over low level implementation details such as data representation, serialization and network protocols. Our ScalaLoci language allows developers to declare the different components and their architectural relation at the type level, allowing static reasoning about about distribution and remote communication and guaranteeing static type safety across components. The compiler splits the multitier program into the component-specific code and automatically generates the communication boilerplate. Communication between components can be modeled by declaratively specifying data flows between components using reactive programming. In this paper, we report on the implementation of our design and our experience with embedding our language features into Scala as a host language. We show how a combination of Scala's advanced type level programming and its macro system can be used to enrich the language with new abstractions. We comment on the challenges we encountered and the solutions we developed for our current implementation and outline suggestions for an improved macro system to support the such use cases of embedding of domain-specific abstractions.
Topology-free immersed boundary method for incompressible turbulence flows: An aerodynamic simulation for 'dirty' CAD geometry
Venue: arXiv cs.CE 2020
Year: 2020

To design a method to solve the issues of handling 'dirty' and highly complex geometries, the topology-free method combined with the immersed boundary method is presented for viscous and incompressible flows at a high Reynolds number. The method simultaneously employs a ghost-cell technique and distributed forcing technique to impose the boundary conditions. An axis-projected interpolation scheme is used to avoid searching failures during fluid and solid identification. This method yields a topology-free immersed boundary, which particularly suits flow simulations of highly complex geometries. Difficulties generally arise when generating the calculation grid for these scenarios. This method allows dirty data to be handled without any preparatory treatment work to simplify or clean-up the geometry. This method is also applicable to the coherent structural turbulence model employed in this study. The verification cases, used in conjunction with the second-order central-difference scheme, resulted in first-order accuracy at finer resolution, although the coarser resolution retained second-order accuracy. This method is fully parallelized for distributed memory platforms. In this study, the accuracy and fidelity of this method were examined by simulating the flow around the bluff body, past a flat plate, and past dirty spheres. These simulations were compared with experimental data and other established results. Finally, results from the simulation of practical applications demonstrate the ability of the method to model highly complex, non-canonical three-dimensional flows. The countermeasure based on the accurate classification of geometric features has provided a robust and reasonable solution.
On State Variables, Bandit Problems and POMDPs
Venue: arXiv cs.LG 2020
Year: 2020

State variables are easily the most subtle dimension of sequential decision problems. This is especially true in the context of active learning problems (bandit problems") where decisions affect what we observe and learn. We describe our canonical framework that models {\it any} sequential decision problem, and present our definition of state variables that allows us to claim: Any properly modeled sequential decision problem is Markovian. We then present a novel two-agent perspective of partially observable Markov decision problems (POMDPs) that allows us to then claim: Any model of a real decision problem is (possibly) non-Markovian. We illustrate these perspectives using the context of observing and treating flu in a population, and provide examples of all four classes of policies in this setting. We close with an indication of how to extend this thinking to multiagent problems.
The Design of Dynamic Probabilistic Caching with Time-Varying Content Popularity
Venue: arXiv cs.NI 2020
Year: 2020

In this paper, we design dynamic probabilistic caching for the scenario when the instantaneous content popularity may vary with time while it is possible to predict the average content popularity over a time window. Based on the average content popularity, optimal content caching probabilities can be found, e.g., from solving optimization problems, and existing results in the literature can implement the optimal caching probabilities via static content placement. The objective of this work is to design dynamic probabilistic caching that: i) converge (in distribution) to the optimal content caching probabilities under time-invariant content popularity, and ii) adapt to the time-varying instantaneous content popularity under time-varying content popularity. Achieving the above objective requires a novel design of dynamic content replacement because static caching cannot adapt to varying content popularity while classic dynamic replacement policies, such as LRU, cannot converge to target caching probabilities (as they do not exploit any content popularity information). We model the design of dynamic probabilistic replacement policy as the problem of finding the state transition probability matrix of a Markov chain and propose a method to generate and refine the transition probability matrix. Extensive numerical results are provided to validate the effectiveness of the proposed design.
Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks
Venue: arXiv cs.AI 2020
Year: 2020

There has been significant progress in recent years in the field of Natural Language Processing thanks to the introduction of the Transformer architecture. Current state-of-the-art models, via a large number of parameters and pre-training on massive text corpus, have shown impressive results on several downstream tasks. Many researchers have studied previous (non-Transformer) models to understand their actual behavior under different scenarios, showing that these models are taking advantage of clues or failures of datasets and that slight perturbations on the input data can severely reduce their performance. In contrast, recent models have not been systematically tested with adversarial-examples in order to show their robustness under severe stress conditions. For that reason, this work evaluates three Transformer-based models (RoBERTa, XLNet, and BERT) in Natural Language Inference (NLI) and Question Answering (QA) tasks to know if they are more robust or if they have the same flaws as their predecessors. As a result, our experiments reveal that RoBERTa, XLNet and BERT are more robust than recurrent neural network models to stress tests for both NLI and QA tasks. Nevertheless, they are still very fragile and demonstrate various unexpected behaviors, thus revealing that there is still room for future improvement in this field.
On Extensions of Maximal Repeats in Compressed Strings
Venue: arXiv cs.DS 2020
Year: 2020

This paper provides an upper bound for several subsets of maximal repeats and maximal pairs in compressed strings and also presents a formerly unknown relationship between maximal pairs and the run-length Burrows-Wheeler transform. This relationship is used to obtain a different proof for the Burrows-Wheeler conjecture which has recently been proven by Kempa and Kociumaka in "Resolution of the Burrows-Wheeler Transform Conjecture". More formally, this paper proves that a string $S$ with $z$ LZ77-factors and without $q$-th powers has at most $73(\log_2 |S|)(z+2)^2$ runs in the run-length Burrows-Wheeler transform and the number of arcs in the compacted directed acyclic word graph of $S$ is bounded from above by $18q(1+\log_q |S|)(z+2)^2$.
Trustworthy AI
Venue: arXiv cs.AI 2020
Year: 2020

The promise of AI is huge. AI systems have already achieved good enough performance to be in our streets and in our homes. However, they can be brittle and unfair. For society to reap the benefits of AI systems, society needs to be able to trust them. Inspired by decades of progress in trustworthy computing, we suggest what trustworthy properties would be desired of AI systems. By enumerating a set of new research questions, we explore one approach--formal verification--for ensuring trust in AI. Trustworthy AI ups the ante on both trustworthy computing and formal methods.
Sparse Coresets for SVD on Infinite Streams
Venue: arXiv cs.DS 2020
Year: 2020

In streaming Singular Value Decomposition (SVD), $d$-dimensional rows of a possibly infinite matrix arrive sequentially as points in $\mathbb{R}^d$. An $ε$-coreset is a (much smaller) matrix whose sum of square distances of the rows to any hyperplane approximates that of the original matrix to a $1 \pm ε$ factor. Our main result is that we can maintain a $ε$-coreset while storing only $O(d \log^2 d / ε^2)$ rows. Known lower bounds of $Ω(d / ε^2)$ rows show that this is nearly optimal. Moreover, each row of our coreset is a weighted subset of the input rows. This is highly desirable since it: (1) preserves sparsity; (2) is easily interpretable; (3) avoids precision errors; (4) applies to problems with constraints on the input. Previous streaming results for SVD that return a subset of the input required storing $Ω(d \log^3 n / ε^2)$ rows where $n$ is the number of rows seen so far. Our algorithm, with storage independent of $n$, is the first result that uses finite memory on infinite streams. We support our findings with experiments on the Wikipedia dataset benchmarked against state-of-the-art algorithms.
Optimal Mechanism Design for Single-Minded Agents
Venue: arXiv cs.GT 2020
Year: 2020

We consider revenue-optimal mechanism design in the interdimensional setting, where one dimension is the 'value' of the buyer, and one is a 'type' that captures some auxiliary information. One setting is the FedEx Problem, for which FGKK [2016] characterize the optimal mechanism for a single agent. We ask: how far can such characterizations go? In particular, we consider single-minded agents. A seller has heterogenous items. A buyer has a value v for a specific subset of items S, and obtains value v iff he gets (at least) all the items in S. We show: 1. Deterministic mechanisms are optimal for distributions that satisfy the "declining marginal revenue" (DMR) property; we give an explicit construction of the optimal mechanism. 2. Without DMR, the result depends on the structure of the directed acyclic graph (DAG) representing the partial order among types. When the DAG has out-degree at most 1, we characterize the optimal mechanism a la FedEx. 3. Without DMR, when the DAG has some node with out-degree at least 2, we show that in this case the menu complexity is unbounded: for any M, there exist distributions over (v,S) pairs such that the menu complexity of the optimal mechanism is at least M. 4. For the case of 3 types, we show that for all distributions there exists an optimal mechanism of finite menu complexity. This is in contrast to 2 additive heterogenous items or which the menu complexity could be uncountable [MV07; DDT15]. In addition, we prove that optimal mechanisms for Multi-Unit Pricing (without DMR) can have unbounded menu complexity. We also propose an extension where the menu complexity of optimal mechanisms can be countable but not uncountable. Together these results establish that optimal mechanisms in interdimensional settings are both much richer than single-dimensional settings, yet also vastly more structured than multi-dimensional settings.
Supervised Phrase-boundary Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

We propose a new word embedding model, called SPhrase, that incorporates supervised phrase information. Our method modifies traditional word embeddings by ensuring that all target words in a phrase have exactly the same context. We demonstrate that including this information within a context window produces superior embeddings for both intrinsic evaluation tasks and downstream extrinsic tasks.
Information-Flow Interfaces
Venue: arXiv cs.FL 2020
Year: 2020

Contract-based design is a promising methodology for taming the complexity of developing sophisticated systems. A formal contract distinguishes between assumptions, which are constraints that the designer of a component puts on the environments in which the component can be used safely, and guarantees, which are promises that the designer asks from the team that implements the component. A theory of formal contracts can be formalized as an interface theory, which supports the composition and refinement of both assumptions and guarantees. Although there is a rich landscape of contract-based design methods that address functional and extra-functional properties, we present the first interface theory that is designed for ensuring system-wide security properties, thus paving the way for a science of safety and security co-engineering. Our framework provides a refinement relation and a composition operation that support both incremental design and independent implementability. We develop our theory for both stateless and stateful interfaces. We illustrate the applicability of our framework with an example inspired from the automotive domain. Finally, we provide three plausible trace semantics to stateful information-flow interfaces and we show that only two correspond to temporal logics for specifying hyperproperties, while the third defines a new class of hyperproperties that lies between the other two classes.
Differentiable Top-k Operator with Optimal Transport
Venue: arXiv cs.LG 2020
Year: 2020

The top-k operation, i.e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining. However, if the top-k operation is implemented in an algorithmic way, e.g., using bubble algorithm, the resulting model cannot be trained in an end-to-end way using prevalent gradient descent algorithms. This is because these implementations typically involve swapping indices, whose gradient cannot be computed. Moreover, the corresponding mapping from the input scores to the indicator vector of whether this element belongs to the top-k set is essentially discontinuous. To address the issue, we propose a smoothed approximation, namely the SOFT (Scalable Optimal transport-based diFferenTiable) top-k operator. Specifically, our SOFT top-k operator approximates the output of the top-k operation as the solution of an Entropic Optimal Transport (EOT) problem. The gradient of the SOFT operator can then be efficiently approximated based on the optimality conditions of EOT problem. We apply the proposed operator to the k-nearest neighbors and beam search algorithms, and demonstrate improved performance.
Exploring Neural Models for Parsing Natural Language into First-Order Logic
Venue: arXiv cs.CL 2020
Year: 2020

Semantic parsing is the task of obtaining machine-interpretable representations from natural language text. We consider one such formal representation - First-Order Logic (FOL) and explore the capability of neural models in parsing English sentences to FOL. We model FOL parsing as a sequence to sequence mapping task where given a natural language sentence, it is encoded into an intermediate representation using an LSTM followed by a decoder which sequentially generates the predicates in the corresponding FOL formula. We improve the standard encoder-decoder model by introducing a variable alignment mechanism that enables it to align variables across predicates in the predicted FOL. We further show the effectiveness of predicting the category of FOL entity - Unary, Binary, Variables and Scoped Entities, at each decoder step as an auxiliary task on improving the consistency of generated FOL. We perform rigorous evaluations and extensive ablations. We also aim to release our code as well as large scale FOL dataset along with models to aid further research in logic-based parsing and inference in NLP.
Automated Labelling using an Attention model for Radiology reports of MRI scans (ALARM)
Venue: arXiv cs.CV 2020
Year: 2020

Labelling large datasets for training high-capacity neural networks is a major obstacle to the development of deep learning-based medical imaging applications. Here we present a transformer-based network for magnetic resonance imaging (MRI) radiology report classification which automates this task by assigning image labels on the basis of free-text expert radiology reports. Our model's performance is comparable to that of an expert radiologist, and better than that of an expert physician, demonstrating the feasibility of this approach. We make code available online for researchers to label their own MRI datasets for medical imaging applications.
Key Points Estimation and Point Instance Segmentation Approach for Lane Detection
Venue: arXiv cs.CV 2020
Year: 2020

Perception techniques for autonomous driving should be adaptive to various environments. In the case of traffic line detection, an essential perception module, many condition should be considered, such as number of traffic lines and computing power of the target system. To address these problems, in this paper, we propose a traffic line detection method called Point Instance Network (PINet); the method is based on the key points estimation and instance segmentation approach. The PINet includes several stacked hourglass networks that are trained simultaneously. Therefore the size of the trained models can be chosen according to the computing power of the target environment. We cast a clustering problem of the predicted key points as an instance segmentation problem; the PINet can be trained regardless of the number of the traffic lines. The PINet achieves competitive accuracy and false positive on the TuSimple and Culane datasets, popular public datasets for lane detection. Our code is available at https://github.com/koyeongmin/PINet_new