OpenRaiser/Intern-Atlas huggingface dataset:
Planar Robot Casting with Real2Sim2Real Self-Supervised Learning
Venue: arXiv cs.RO 2022
Year: 2022

This paper introduces the task of {\em Planar Robot Casting (PRC)}: where one planar motion of a robot arm holding one end of a cable causes the other end to slide across the plane toward a desired target. PRC allows the cable to reach points beyond the robot workspace and has applications for cable management in homes, warehouses, and factories. To efficiently learn a PRC policy for a given cable, we propose Real2Sim2Real, a self-supervised framework that automatically collects physical trajectory examples to tune parameters of a dynamics simulator using Differential Evolution, generates many simulated examples, and then learns a policy using a weighted combination of simulated and physical data. We evaluate Real2Sim2Real with three simulators, Isaac Gym-segmented, Isaac Gym-hybrid, and PyBullet, two function approximators, Gaussian Processes and Neural Networks (NNs), and three cables with differing stiffness, torsion, and friction. Results with 240 physical trials suggest that the PRC policies can attain median error distance (as % of cable length) ranging from 8% to 14%, outperforming baselines and policies trained on only real or only simulated examples. Code, data, and videos are available at https://tinyurl.com/robotcast.
Safe Data Collection for Offline and Online Policy Learning
Venue: arXiv cs.LG 2022
Year: 2022

Motivated by practical needs of experimentation and policy learning in online platforms, we study the problem of safe data collection. Specifically, our goal is to develop a logging policy that efficiently explores different actions to elicit information while achieving competitive reward with a baseline production policy. We first show that a common practice of mixing the production policy with randomized exploration, despite being safe, is sub-optimal in maximizing information gain. Then, we propose a safe optimal logging policy via a novel water-filling technique for the case when no side information about the actions' expected reward is available. We improve upon this design by considering side information and also extend our approaches to the linear contextual model to account for a large number of actions. Along the way, we analyze how our data logging policies impact errors in off(line)-policy learning and empirically validate the benefit of our design by conducting extensive numerical experiments with synthetic and MNIST datasets. To further demonstrate the generality of our approach, we also consider the safe online learning setting. By adaptively applying our techniques, we develop the Safe Phased-Elimination (SafePE) algorithm that can achieve optimal regret bound with only logarithmic number of policy updates.
Papaya: Practical, Private, and Scalable Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Cross-device Federated Learning (FL) is a distributed learning paradigm with several challenges that differentiate it from traditional distributed learning, variability in the system characteristics on each device, and millions of clients coordinating with a central server being primary ones. Most FL systems described in the literature are synchronous - they perform a synchronized aggregation of model updates from individual clients. Scaling synchronous FL is challenging since increasing the number of clients training in parallel leads to diminishing returns in training speed, analogous to large-batch training. Moreover, stragglers hinder synchronous FL training. In this work, we outline a production asynchronous FL system design. Our work tackles the aforementioned issues, sketches of some of the system design challenges and their solutions, and touches upon principles that emerged from building a production FL system for millions of clients. Empirically, we demonstrate that asynchronous FL converges faster than synchronous FL when training across nearly one hundred million devices. In particular, in high concurrency settings, asynchronous FL is 5x faster and has nearly 8x less communication overhead than synchronous FL.
Graph-Based Depth Denoising & Dequantization for Point Cloud Enhancement
Venue: arXiv cs.CV 2022
Year: 2022

A 3D point cloud is typically constructed from depth measurements acquired by sensors at one or more viewpoints. The measurements suffer from both quantization and noise corruption. To improve quality, previous works denoise a point cloud \textit{a posteriori} after projecting the imperfect depth data onto 3D space. Instead, we enhance depth measurements directly on the sensed images \textit{a priori}, before synthesizing a 3D point cloud. By enhancing near the physical sensing process, we tailor our optimization to our depth formation model before subsequent processing steps that obscure measurement errors. Specifically, we model depth formation as a combined process of signal-dependent noise addition and non-uniform log-based quantization. The designed model is validated (with parameters fitted) using collected empirical data from a representative depth sensor. To enhance each pixel row in a depth image, we first encode intra-view similarities between available row pixels as edge weights via feature graph learning. We next establish inter-view similarities with another rectified depth image via viewpoint mapping and sparse linear interpolation. This leads to a maximum a posteriori (MAP) graph filtering objective that is convex and differentiable. We minimize the objective efficiently using accelerated gradient descent (AGD), where the optimal step size is approximated via Gershgorin circle theorem (GCT). Experiments show that our method significantly outperformed recent point cloud denoising schemes and state-of-the-art image denoising schemes in two established point cloud quality metrics.
View Birdification in the Crowd: Ground-Plane Localization from Perceived Movements
Venue: arXiv cs.CV 2022
Year: 2022

We introduce view birdification, the problem of recovering ground-plane movements of people in a crowd from an ego-centric video captured from an observer (e.g., a person or a vehicle) also moving in the crowd. Recovered ground-plane movements would provide a sound basis for situational understanding and benefit downstream applications in computer vision and robotics. In this paper, we formulate view birdification as a geometric trajectory reconstruction problem and derive a cascaded optimization method from a Bayesian perspective. The method first estimates the observer's movement and then localizes surrounding pedestrians for each frame while taking into account the local interactions between them. We introduce three datasets by leveraging synthetic and real trajectories of people in crowds and evaluate the effectiveness of our method. The results demonstrate the accuracy of our method and set the ground for further studies of view birdification as an important but challenging visual understanding problem.
Residual Quantity in Percentage of Factory Machines Using Computer Vision and Mathematical Methods
Venue: arXiv cs.CV 2022
Year: 2022

Computer vision has been thriving since AI development was gaining thrust. Using deep learning techniques has been the most popular way which computer scientists thought the solution of. However, deep learning techniques tend to show lower performance than manual processing. Using deep learning is not always the answer to a problem related to computer vision.
BreakBot: Analyzing the Impact of Breaking Changes to Assist Library Evolution
Venue: arXiv cs.SE 2022
Year: 2022

"If we make this change to our code, how will it impact our clients?" It is difficult for library maintainers to answer this simple-yet essential!-question when evolving their libraries. Library maintainers are constantly balancing between two opposing positions: make changes at the risk of breaking some of their clients, or avoid changes and maintain compatibility at the cost of immobility and growing technical debt. We argue that the lack of objective usage data and tool support leaves maintainers with their own subjective perception of their community to make these decisions. We introduce BreakBot, a bot that analyses the pull requests of Java libraries on GitHub to identify the breaking changes they introduce and their impact on client projects. Through static analysis of libraries and clients, it extracts and summarizes objective data that enrich the code review process by providing maintainers with the appropriate information to decide whether-and how-changes should be accepted, directly in the pull requests.
DataWords: Getting Contrarian with Text, Structured Data and Explanations
Venue: arXiv cs.LG 2022
Year: 2022

Our goal is to build classification models using a combination of free-text and structured data. To do this, we represent structured data by text sentences, DataWords, so that similar data items are mapped into the same sentence. This permits modeling a mixture of text and structured data by using only text-modeling algorithms. Several examples illustrate that it is possible to improve text classification performance by first running extraction tools (named entity recognition), then converting the output to DataWords, and adding the DataWords to the original text -- before model building and classification. This approach also allows us to produce explanations for inferences in terms of both free text and structured data.
Complexity of the Ackermann fragment with one leading existential quantifier
Venue: arXiv cs.LO 2022
Year: 2022

In this short note we prove that the satisfiability problem of the Ackermann fragment with one leading existential quantifier is ExpTime-complete.
ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guarantees
Venue: arXiv cs.LG 2022
Year: 2022

Models recently used in the literature proving residual networks (ResNets) are better than linear predictors are actually different from standard ResNets that have been widely used in computer vision. In addition to the assumptions such as scalar-valued output or single residual block, these models have no nonlinearities at the final residual representation that feeds into the final affine layer. To codify such a difference in nonlinearities and reveal a linear estimation property, we define ResNEsts, i.e., Residual Nonlinear Estimators, by simply dropping nonlinearities at the last residual representation from standard ResNets. We show that wide ResNEsts with bottleneck blocks can always guarantee a very desirable training property that standard ResNets aim to achieve, i.e., adding more blocks does not decrease performance given the same set of basis elements. To prove that, we first recognize ResNEsts are basis function models that are limited by a coupling problem in basis learning and linear prediction. Then, to decouple prediction weights from basis learning, we construct a special architecture termed augmented ResNEst (A-ResNEst) that always guarantees no worse performance with the addition of a block. As a result, such an A-ResNEst establishes empirical risk lower bounds for a ResNEst using corresponding bases. Our results demonstrate ResNEsts indeed have a problem of diminishing feature reuse; however, it can be avoided by sufficiently expanding or widening the input space, leading to the above-mentioned desirable property. Inspired by the DenseNets that have been shown to outperform ResNets, we also propose a corresponding new model called Densely connected Nonlinear Estimator (DenseNEst). We show that any DenseNEst can be represented as a wide ResNEst with bottleneck blocks. Unlike ResNEsts, DenseNEsts exhibit the desirable property without any special architectural re-design.
Attention Approximates Sparse Distributed Memory
Venue: arXiv cs.LG 2022
Year: 2022

While Attention has come to be an important mechanism in deep learning, there remains limited intuition for why it works so well. Here, we show that Transformer Attention can be closely related under certain data conditions to Kanerva's Sparse Distributed Memory (SDM), a biologically plausible associative memory model. We confirm that these conditions are satisfied in pre-trained GPT2 Transformer models. We discuss the implications of the Attention-SDM map and provide new computational and biological interpretations of Attention.
Palette: Image-to-Image Diffusion Models
Venue: arXiv cs.CV 2022
Year: 2022

This paper develops a unified framework for image-to-image translation based on conditional diffusion models and evaluates this framework on four challenging image-to-image translation tasks, namely colorization, inpainting, uncropping, and JPEG restoration. Our simple implementation of image-to-image diffusion models outperforms strong GAN and regression baselines on all tasks, without task-specific hyper-parameter tuning, architecture customization, or any auxiliary loss or sophisticated new techniques needed. We uncover the impact of an L2 vs. L1 loss in the denoising diffusion objective on sample diversity, and demonstrate the importance of self-attention in the neural architecture through empirical studies. Importantly, we advocate a unified evaluation protocol based on ImageNet, with human evaluation and sample quality scores (FID, Inception Score, Classification Accuracy of a pre-trained ResNet-50, and Perceptual Distance against original images). We expect this standardized evaluation protocol to play a role in advancing image-to-image translation research. Finally, we show that a generalist, multi-task diffusion model performs as well or better than task-specific specialist counterparts. Check out https://diffusion-palette.github.io for an overview of the results.
Multi-Objective Optimization for Value-Sensitive and Sustainable Basket Recommendations
Venue: arXiv cs.NE 2022
Year: 2022

Sustainable consumption aims to minimize the environmental and societal impact of the use of services and products. Over-consumption of services and products leads to potential natural resource exhaustion and societal inequalities as access to goods and services becomes more challenging. In everyday life, a person can simply achieve more sustainable purchases by drastically changing their lifestyle choices and potentially going against their personal values or wishes. Conversely, achieving sustainable consumption while accounting for personal values is a more complex task as potential trade-offs arise when trying to satisfy environmental and personal goals. This article focuses on value-sensitive design of recommender systems, which enable consumers to improve the sustainability of their purchases while respecting personal and societal values. Value-sensitive recommendations for sustainable consumption are formalized as a multi-objective optimization problem, where each objective represents different sustainability goals and personal values. Novel and existing multi-objective algorithms calculate solutions to this problem. The solutions are proposed as personalized sustainable basket recommendations to consumers. These recommendations are evaluated on a synthetic dataset, which comprises three established real-world datasets from relevant scientific and organizational reports. The synthetic dataset contains quantitative data on product prices, nutritional values, and environmental impact metrics, such as greenhouse gas emissions and water footprint. The recommended baskets are highly similar to consumer purchased baskets and aligned with both sustainability goals and personal values relevant to health, expenditure, and taste. Even when consumers would accept only a fraction of recommendations, a considerable reduction of environmental impact is observed.
Grand-potential-based phase-field model of dissolution/precipitation: lattice Boltzmann simulations of counter term effect on porous medium
Venue: arXiv cs.CE 2022
Year: 2022

Most of the lattice Boltzmann methods simulate an approximation of the sharp interface problem of dissolution and precipitation. In such studies the curvature-driven motion of interface is neglected in the Gibbs-Thomson condition. In order to simulate those phenomena with or without curvature-driven motion, we propose a phase-field model which is derived from a thermodynamic functional of grand-potential. Compared to the free energy, the main advantage of the grand-potential is to provide a theoretical framework which is consistent with the equilibrium properties such as the equality of chemical potentials. The model is composed of one equation for the phase-field ϕ coupled with one equation for the chemical potential μ. In the phase-field method, the curvature-driven motion is always contained in the phase-field equation. For canceling it, a counter term must be added in the ϕ-equation. For reason of mass conservation, the μ-equation is written with a mixed formulation which involves the composition c and the chemical potential. The closure relationship between c and μ is derived by assuming quadratic free energies of bulk phases. The anti-trapping current is also considered in the composition equation for simulations with null solid diffusion. The lattice Boltzmann schemes are implemented in LBM_saclay, a numerical code running on various High Performance Computing architectures. Validations are carried out with analytical solutions representative of dissolution and precipitation. Simulations with or without counter term are compared on the shape of porous medium characterized by microtomography. The computations have run on a single GPU-V100.
Linear Speedup in Personalized Collaborative Learning
Venue: arXiv cs.LG 2022
Year: 2022

Collaborative training can improve the accuracy of a model for a user by trading off the model's bias (introduced by using data from other users who are potentially different) against its variance (due to the limited amount of data on any single user). In this work, we formalize the personalized collaborative learning problem as a stochastic optimization of a task 0 while giving access to N related but different tasks 1,..., N. We provide convergence guarantees for two algorithms in this setting -- a popular collaboration method known as weighted gradient averaging, and a novel bias correction method -- and explore conditions under which we can achieve linear speedup w.r.t. the number of auxiliary tasks N. Further, we also empirically study their performance confirming our theoretical insights.
ConTesse: Accurate Occluding Contours for Subdivision Surfaces
Venue: arXiv cs.GR 2022
Year: 2022

This paper proposes a method for computing the visible occluding contours of subdivision surfaces. The paper first introduces new theory for contour visibility of smooth surfaces. Necessary and sufficient conditions are introduced for when a sampled occluding contour is valid, that is, when it may be assigned consistent visibility. Previous methods do not guarantee these conditions, which helps explain why smooth contour visibility has been such a challenging problem in the past. The paper then proposes an algorithm that, given a subdivision surface, finds sampled contours satisfying these conditions, and then generates a new triangle mesh matching the given occluding contours. The contours of the output triangle mesh may then be rendered with standard non-photorealistic rendering algorithms, using the mesh for visibility computation. The method can be applied to any triangle mesh, by treating it as the base mesh of a subdivision surface.
Improving Novelty Detection using the Reconstructions of Nearest Neighbours
Venue: arXiv cs.LG 2022
Year: 2022

We show that using nearest neighbours in the latent space of autoencoders (AE) significantly improves performance of semi-supervised novelty detection in both single and multi-class contexts. Autoencoding methods detect novelty by learning to differentiate between the non-novel training class(es) and all other unseen classes. Our method harnesses a combination of the reconstructions of the nearest neighbours and the latent-neighbour distances of a given input's latent representation. We demonstrate that our nearest-latent-neighbours (NLN) algorithm is memory and time efficient, does not require significant data augmentation, nor is reliant on pre-trained networks. Furthermore, we show that the NLN-algorithm is easily applicable to multiple datasets without modification. Additionally, the proposed algorithm is agnostic to autoencoder architecture and reconstruction error method. We validate our method across several standard datasets for a variety of different autoencoding architectures such as vanilla, adversarial and variational autoencoders using either reconstruction, residual or feature consistent losses. The results show that the NLN algorithm grants up to a 17% increase in Area Under the Receiver Operating Characteristics (AUROC) curve performance for the multi-class case and 8% for single-class novelty detection.
Queueing Theoretic Models for Multiuser MISO Content-Centric Networks with SDMA, NOMA, OMA and Rate-Splitting Downlink
Venue: arXiv cs.NI 2022
Year: 2022

Multiuser, Multiple Input, Single Output (MU-MISO) systems are proving to be indispensable in the next generation wireless networks such as 5G and 6G. The spatial diversity of MISO systems have been leveraged in physical layer designs in these wireless systems to improve the capacity. Several recent studies have utilised redundancies in the content request along with the spatial diversity of a MISO system to improve the capacity further. It is shown that Max-Min Fair (MMF) Beamforming schemes for MISO based on SDMA, NOMA, OMA and Rate-Splitting could be used to improve the content delivery rates. However, in most of these studies the key aspects such as the queueing delays in the downlink and the user dynamics have generally been ignored. In this work, we study how the interplay between queueing, beamforming and the user dynamics affects the Quality-of-Service (user experienced delay) of downlink in MU-MISO content centric networks (CCNs). We propose queueing theoretic models that are simple in nature and can be directly adapted to MU-MISO CCNs to perform optimal multi-group multicast downlink transmissions. We show that a recently developed Simple Multicast Queue (SMQ) for SISO systems can be directly used for MU-MISO systems and that it provides superior performance due to its always-stable nature. Further, we observe that MMF Beamforming schemes coupled with SMQ can be quite unfair to users with good channels. Thus, we propose an improvement to SMQ called Dual SMQ which addresses this issue. We also provide theoretical analysis of the mean delay experienced by the users in such MU-MISO CCNs.