OpenRaiser/Intern-Atlas huggingface dataset:
Learning Models over Relational Data using Sparse Tensors and Functional Dependencies
Venue: arXiv cs.DB 2020
Year: 2020

Integrated solutions for analytics over relational databases are of great practical importance as they avoid the costly repeated loop data scientists have to deal with on a daily basis: select features from data residing in relational databases using feature extraction queries involving joins, projections, and aggregations; export the training dataset defined by such queries; convert this dataset into the format of an external learning tool; and train the desired model using this tool. These integrated solutions are also a fertile ground of theoretically fundamental and challenging problems at the intersection of relational and statistical data models. This article introduces a unified framework for training and evaluating a class of statistical learning models over relational databases. This class includes ridge linear regression, polynomial regression, factorization machines, and principal component analysis. We show that, by synergizing key tools from database theory such as schema information, query structure, functional dependencies, recent advances in query evaluation algorithms, and from linear algebra such as tensor and matrix operations, one can formulate relational analytics problems and design efficient (query and data) structure-aware algorithms to solve them. This theoretical development informed the design and implementation of the AC/DC system for structure-aware learning. We benchmark the performance of AC/DC against R, MADlib, libFM, and TensorFlow. For typical retail forecasting and advertisement planning applications, AC/DC can learn polynomial regression models and factorization machines with at least the same accuracy as its competitors and up to three orders of magnitude faster than its competitors whenever they do not run out of memory, exceed 24-hour timeout, or encounter internal design limitations.
Chance-Constrained Unit Commitment with N-1 Security and Wind Uncertainty
Venue: arXiv cs.SY 2020
Year: 2020

As renewable wind energy penetration rates continue to increase, one of the major challenges facing grid operators is the question of how to control transmission grids in a reliable and a cost-efficient manner. The stochastic nature of wind forces an alteration of traditional methods for solving day-ahead and look-ahead unit commitment and dispatch. In particular, the variability of wind generation increases the risk of unexpected overloads and cascading events. To address these questions, we present an N-1 Security and Chance-Constrained Unit Commitment (SCCUC) that includes models of generation reserves that respond to wind fluctuations and component outages. We formulate the SCCUC as a mixed-integer, second-order cone problem that limits the probability of failure. We develop a modified Benders decomposition algorithm to solve the problem to optimality and present detailed case studies on the IEEE RTS-96 three-area and the IEEE 300 NESTA test systems. The case studies assess the economic impacts of contingencies and various degrees of wind power penetration and demonstrate the effectiveness and scalability of the algorithm.
Process algebra with strategic interleaving
Venue: arXiv cs.LO 2020
Year: 2020

In process algebras such as ACP (Algebra of Communicating Processes), parallel processes are considered to be interleaved in an arbitrary way. In the case of multi-threading as found in contemporary programming languages, parallel processes are actually interleaved according to some interleaving strategy. An interleaving strategy is what is called a process-scheduling policy in the field of operating systems. In many systems, for instance hardware/software systems, we have to do with both parallel processes that may best be considered to be interleaved in an arbitrary way and parallel processes that may best be considered to be interleaved according to some interleaving strategy. Therefore, we extend ACP in this paper with the latter form of interleaving. The established properties of the extension concerned include an elimination property, a conservative extension property, and a unique expansion property.
Randomized CP Tensor Decomposition
Venue: arXiv cs.NA 2020
Year: 2020

The CANDECOMP/PARAFAC (CP) tensor decomposition is a popular dimensionality-reduction method for multiway data. Dimensionality reduction is often sought after since many high-dimensional tensors have low intrinsic rank relative to the dimension of the ambient measurement space. However, the emergence of `big data' poses significant computational challenges for computing this fundamental tensor decomposition. By leveraging modern randomized algorithms, we demonstrate that coherent structures can be learned from a smaller representation of the tensor in a fraction of the time. Thus, this simple but powerful algorithm enables one to compute the approximate CP decomposition even for massive tensors. The approximation error can thereby be controlled via oversampling and the computation of power iterations. In addition to theoretical results, several empirical results demonstrate the performance of the proposed algorithm.
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Venue: arXiv cs.CV 2020
Year: 2020

Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs. However, for many tasks, paired training data will not be available. We present an approach for learning to translate an image from a source domain $X$ to a target domain $Y$ in the absence of paired examples. Our goal is to learn a mapping $G: X \rightarrow Y$ such that the distribution of images from $G(X)$ is indistinguishable from the distribution $Y$ using an adversarial loss. Because this mapping is highly under-constrained, we couple it with an inverse mapping $F: Y \rightarrow X$ and introduce a cycle consistency loss to push $F(G(X)) \approx X$ (and vice versa). Qualitative results are presented on several tasks where paired training data does not exist, including collection style transfer, object transfiguration, season transfer, photo enhancement, etc. Quantitative comparisons against several prior methods demonstrate the superiority of our approach.
The Dynamics of Rank-Maximal and Popular Matchings
Venue: arXiv cs.DS 2020
Year: 2020

Given a bipartite graph, where the two sets of vertices are applicants and posts and ranks on the edges represent preferences of applicants over posts, a {\em rank-maximal} matching is one in which the maximum number of applicants is matched to their rank one posts and subject to this condition, the maximum number of applicants is matched to their rank two posts, and so on. We study the dynamic version of the problem in which a new applicant or post may be added to the graph and we would like to maintain a rank-maximal matching. We show that after the arrival of one vertex, we are always able to update the existing rank-maximal matching in $\mathcal{O}(\min(c'n ,n^2) + m)$ time, where $n$ denotes the number of applicants, $m$ the number of edges and $c'$ the maximum rank of an edge in an optimal solution. Additionally, we update the matching using a minimal number of changes (replacements). All cases of a deletion of a vertex/edge and an addition of an edge can be reduced to the problem of handling the addition of a vertex. As a by-product, we also get an analogous $\mathcal{O}(m)$ result for the dynamic version of the (one-sided) popular matching problem. Our results are based on the novel use of the properties of the Edmonds-Gallai decomposition. The presented ideas may find applications in other (dynamic) matching problems.
Preimage problems for deterministic finite automata
Venue: arXiv cs.FL 2020
Year: 2020

Given a subset of states $S$ of a deterministic finite automaton and a word $w$, the preimage is the subset of all states mapped to a state in $S$ by the action of $w$. We study three natural problems concerning words giving certain preimages. The first problem is whether, for a given subset, there exists a word \emph{extending} the subset (giving a larger preimage). The second problem is whether there exists a \emph{totally extending} word (giving the whole set of states as a preimage)---equivalently, whether there exists an \emph{avoiding} word for the complementary subset. The third problem is whether there exists a \emph{resizing} word. We also consider variants where the length of the word is upper bounded, where the size of the given subset is restricted, and where the automaton is strongly connected, synchronizing, or binary. We conclude with a summary of the complexities in all combinations of the cases.
Semi-Global Weighted Least Squares in Image Filtering
Venue: arXiv cs.GR 2020
Year: 2020

Solving the global method of Weighted Least Squares (WLS) model in image filtering is both time- and memory-consuming. In this paper, we present an alternative approximation in a time- and memory- efficient manner which is denoted as Semi-Global Weighed Least Squares (SG-WLS). Instead of solving a large linear system, we propose to iteratively solve a sequence of subsystems which are one-dimensional WLS models. Although each subsystem is one-dimensional, it can take two-dimensional neighborhood information into account due to the proposed special neighborhood construction. We show such a desirable property makes our SG-WLS achieve close performance to the original two-dimensional WLS model but with much less time and memory cost. While previous related methods mainly focus on the 4-connected/8-connected neighborhood system, our SG-WLS can handle a more general and larger neighborhood system thanks to the proposed fast solution. We show such a generalization can achieve better performance than the 4-connected/8-connected neighborhood system in some applications. Our SG-WLS is $\sim20$ times faster than the WLS model. For an image of $M\times N$, the memory cost of SG-WLS is at most at the magnitude of $max\{\frac{1}{M}, \frac{1}{N}\}$ of that of the WLS model. We show the effectiveness and efficiency of our SG-WLS in a range of applications. The code is publicly available at: https://github.com/wliusjtu/Semi-Global-Weighted-Least-Squares-in-Image-Filtering.
Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space -- Fundamental Theory and Methods
Venue: arXiv cs.AI 2020
Year: 2020

Policy iteration (PI) is a recursive process of policy evaluation and improvement for solving an optimal decision-making/control problem, or in other words, a reinforcement learning (RL) problem. PI has also served as the fundamental for developing RL methods. In this paper, we propose two PI methods, called differential PI (DPI) and integral PI (IPI), and their variants, for a general RL framework in continuous time and space (CTS), where the environment is modeled by a system of ordinary differential equations (ODEs). The proposed methods inherit the current ideas of PI in classical RL and optimal control and theoretically support the existing RL algorithms in CTS: TD-learning and value-gradient-based (VGB) greedy policy update. We also provide case studies including 1) discounted RL and 2) optimal control tasks. Fundamental mathematical properties -- admissibility, uniqueness of the solution to the Bellman equation (BE), monotone improvement, convergence, and optimality of the solution to the Hamilton-Jacobi-Bellman equation (HJBE) -- are all investigated in-depth and improved from the existing theory, along with the general and case studies. Finally, the proposed ones are simulated with an inverted-pendulum model and their model-based and partially model-free implementations to support the theory and further investigate them beyond.
Model-based Catheter Segmentation in MRI-images
Venue: arXiv cs.CV 2020
Year: 2020

Accurate and reliable segmentation of catheters in MR-guided interventions remains a challenge, and a step of critical importance in clinical workflows. In this work, under reasonable assumptions, mechanical model based heuristics guide the segmentation process allows correct catheter identification rates greater than 98% (error 2.88 mm), and reduction in outliers to one-fourth compared to the state of the art. Given distal tips, searching towards the proximal ends of the catheters is guided by mechanical models that are estimated on a per-catheter basis. Their bending characteristics are used to constrain the image feature based candidate points. The final catheter trajectories are hybrid sequences of individual points, each derived from model and image features. We evaluate the method on a database of 10 patient MRI scans including 101 manually segmented catheters. The mean errors were 1.40 mm and the median errors were 1.05 mm. The number of outliers deviating more than 2 mm from the gold standard is 7, and the number of outliers deviating more than 3 mm from the gold standard is just 2.
Posterior sampling for reinforcement learning: worst-case regret bounds
Venue: arXiv cs.LG 2020
Year: 2020

We present an algorithm based on posterior sampling (aka Thompson sampling) that achieves near-optimal worst-case regret bounds when the underlying Markov Decision Process (MDP) is communicating with a finite, though unknown, diameter. Our main result is a high probability regret upper bound of $\tilde{O}(DS\sqrt{AT})$ for any communicating MDP with $S$ states, $A$ actions and diameter $D$. Here, regret compares the total reward achieved by the algorithm to the total expected reward of an optimal infinite-horizon undiscounted average reward policy, in time horizon $T$. This result closely matches the known lower bound of $Ω(\sqrt{DSAT})$. Our techniques involve proving some novel results about the anti-concentration of Dirichlet distribution, which may be of independent interest.
Convergence of backpropagation with momentum for network architectures with skip connections
Venue: arXiv cs.CV 2020
Year: 2020

We study a class of deep neural networks with networks that form a directed acyclic graph (DAG). For backpropagation defined by gradient descent with adaptive momentum, we show weights converge for a large class of nonlinear activation functions. The proof generalizes the results of Wu et al. (2008) who showed convergence for a feed forward network with one hidden layer. For an example of the effectiveness of DAG architectures, we describe an example of compression through an autoencoder, and compare against sequential feed forward networks under several metrics.
Capacity Scaling of Cellular Networks: Impact of Bandwidth, Infrastructure Density and Number of Antennas
Venue: arXiv cs.IT 2020
Year: 2020

The availability of very wide spectrum in millimeter wave bands combined with large antenna arrays and ultra dense networks raises two basic questions: What is the true value of overly abundant degrees of freedom and how can networks be designed to fully exploit them? This paper determines the capacity scaling of large cellular networks as a function of bandwidth, area, number of antennas and base station density. It is found that the network capacity has a fundamental bandwidth scaling limit, beyond which the network becomes power-limited. An infrastructure multi-hop protocol achieves the optimal network capacity scaling for all network parameters. In contrast, current protocols that use only single-hop direct transmissions can not achieve the capacity scaling in wideband regimes except in the special case when the density of base stations is taken to impractical extremes. This finding suggests that multi-hop communication will be important to fully realize the potential of next-generation cellular networks. Dedicated relays, if sufficiently dense, can also perform this task, relieving user nodes from the battery drain of cooperation. On the other hand, more sophisticated strategies such as hierarchical cooperation, that are essential for achieving capacity scaling in ad hoc networks, are unnecessary in the cellular context.
Image Compression Based on Compressive Sensing: End-to-End Comparison with JPEG
Venue: arXiv cs.CV 2020
Year: 2020

We present an end-to-end image compression system based on compressive sensing. The presented system integrates the conventional scheme of compressive sampling and reconstruction with quantization and entropy coding. The compression performance, in terms of decoded image quality versus data rate, is shown to be comparable with JPEG and significantly better at the low rate range. We study the parameters that influence the system performance, including (i) the choice of sensing matrix, (ii) the trade-off between quantization and compression ratio, and (iii) the reconstruction algorithms. We propose an effective method to jointly control the quantization step and compression ratio in order to achieve near optimal quality at any given bit rate. Furthermore, our proposed image compression system can be directly used in the compressive sensing camera, e.g. the single pixel camera, to construct a hardware compressive sampling system.
Rapid Probabilistic Interest Learning from Domain-Specific Pairwise Image Comparisons
Venue: arXiv cs.CV 2020
Year: 2020

A great deal of work aims to discover large general purpose models of image interest or memorability for visual search and information retrieval. This paper argues that image interest is often domain and user specific, and that efficient mechanisms for learning about this domain-specific image interest as quickly as possible, while limiting the amount of data-labelling required, are often more useful to end-users. This work uses pairwise image comparisons to reduce the labelling burden on these users, and introduces an image interest estimation approach that performs similarly to recent data hungry deep learning approaches trained using pairwise ranking losses. Here, we use a Gaussian process model to interpolate image interest inferred using a Bayesian ranking approach over image features extracted using a pre-trained convolutional neural network. Results show that fitting a Gaussian process in high-dimensional image feature space is not only computationally feasible, but also effective across a broad range of domains. The proposed probabilistic interest estimation approach produces image interests paired with uncertainties that can be used to identify images for which additional labelling is required and measure inference convergence, allowing for sample efficient active model training. Importantly, the probabilistic formulation allows for effective visual search and information retrieval when limited labelling data is available.
Rotational Rectification Network: Enabling Pedestrian Detection for Mobile Vision
Venue: arXiv cs.CV 2020
Year: 2020

Across a majority of pedestrian detection datasets, it is typically assumed that pedestrians will be standing upright with respect to the image coordinate system. This assumption, however, is not always valid for many vision-equipped mobile platforms such as mobile phones, UAVs or construction vehicles on rugged terrain. In these situations, the motion of the camera can cause images of pedestrians to be captured at extreme angles. This can lead to very poor pedestrian detection performance when using standard pedestrian detectors. To address this issue, we propose a Rotational Rectification Network (R2N) that can be inserted into any CNN-based pedestrian (or object) detector to adapt it to significant changes in camera rotation. The rotational rectification network uses a 2D rotation estimation module that passes rotational information to a spatial transformer network to undistort image features. To enable robust rotation estimation, we propose a Global Polar Pooling (GP-Pooling) operator to capture rotational shifts in convolutional features. Through our experiments, we show how our rotational rectification network can be used to improve the performance of the state-of-the-art pedestrian detector under heavy image rotation by up to 45%
Common Counterfactual Belief of Rationality Subsumes Superrationality On Symmetric Games
Venue: arXiv cs.GT 2020
Year: 2020

This paper shows that, for symmetric games in normal form, strategy profiles that satisfy Hofstadter's Superrationality criterion also satisfy both of Halpern's and Pass's criteria under Common Counterfactual Belief of Rationality: minimax-rationalizability and individual rationality.
A Two-Stage Architecture for Differentially Private Kalman Filtering and LQG Control
Venue: arXiv cs.SY 2020
Year: 2020

Large-scale monitoring and control systems enabling a more intelligent infrastructure increasingly rely on sensitive data obtained from private agents, e.g., location traces collected from the users of an intelligent transportation system. In order to encourage the participation of these agents, it becomes then critical to design algorithms that process information in a privacy-preserving way. This article revisits the Kalman filtering and Linear Quadratic Gaussian (LQG) control problems, subject to privacy constraints. We aim to enforce differential privacy, a formal, state-of-the-art definition of privacy ensuring that the output of an algorithm is not too sensitive to the data collected from any single participating agent. A two-stage architecture is proposed that first aggregates and combines the individual agent signals before adding privacy-preserving noise and post-filtering the result to be published. We show a significant performance improvement offered by this architecture over input perturbation schemes as the number of input signals increases and that an optimal static aggregation stage can be computed by solving a semidefinite program. The two-stage architecture, which we develop first for Kalman filtering, is then adapted to the LQG control problem by leveraging the separation principle. Numerical simulations illustrate the performance improvements over differentially private algorithms without first-stage signal aggregation.
DROP: Dimensionality Reduction Optimization for Time Series
Venue: arXiv cs.DB 2020
Year: 2020

Dimensionality reduction is a critical step in scaling machine learning pipelines. Principal component analysis (PCA) is a standard tool for dimensionality reduction, but performing PCA over a full dataset can be prohibitively expensive. As a result, theoretical work has studied the effectiveness of iterative, stochastic PCA methods that operate over data samples. However, termination conditions for stochastic PCA either execute for a predetermined number of iterations, or until convergence of the solution, frequently sampling too many or too few datapoints for end-to-end runtime improvements. We show how accounting for downstream analytics operations during DR via PCA allows stochastic methods to efficiently terminate after operating over small (e.g., 1%) subsamples of input data, reducing whole workload runtime. Leveraging this, we propose DROP, a DR optimizer that enables speedups of up to 5x over Singular-Value-Decomposition-based PCA techniques, and exceeds conventional approaches like FFT and PAA by up to 16x in end-to-end workloads.