OpenRaiser/Intern-Atlas huggingface dataset:
Fast yet predictable braking manoeuvers for real-time robot control
Venue: arXiv cs.RO 2023
Year: 2023

This paper proposes a framework for generating fast, smooth and predictable braking manoeuvers for a controlled robot. The proposed framework integrates two approaches to obtain feasible modal limits for designing braking trajectories. The first approach is real-time capable but conservative considering the usage of the available feasible actuator control region, resulting in longer braking times. In contrast, the second approach maximizes the used braking control inputs at the cost of requiring more time to evaluate larger, feasible modal limits via optimization. Both approaches allow for predicting the robot's stopping trajectory online. In addition, we also formulated and solved a constrained, nonlinear final-time minimization problem to find optimal torque inputs. The optimal solutions were used as a benchmark to evaluate the performance of the proposed predictable braking framework. A comparative study was compiled in simulation versus a classical optimal controller on a 7-DoF robot arm with only three moving joints. The results verified the effectiveness of our proposed framework and its integrated approaches in achieving fast robot braking manoeuvers with accurate online predictions of the stopping trajectories and distances under various braking settings.
Push: Concurrent Probabilistic Programming for Bayesian Deep Learning
Venue: arXiv cs.LG 2023
Year: 2023

We introduce a library called Push that takes a probabilistic programming approach to Bayesian deep learning (BDL). This library enables concurrent execution of BDL inference algorithms on multi-GPU hardware for neural network (NN) models. To accomplish this, Push introduces an abstraction that represents an input NN as a particle. Push enables easy creation of particles so that an input NN can be replicated and particles can communicate asynchronously so that a variety of parameter updates can be expressed, including common BDL algorithms. Our hope is that Push lowers the barrier to experimenting with BDL by streamlining the scaling of particles across GPUs. We evaluate the scaling behavior of particles on single-node multi-GPU devices on vision and scientific machine learning (SciML) tasks.
MANER: Multi-Agent Neural Rearrangement Planning of Objects in Cluttered Environments
Venue: arXiv cs.RO 2023
Year: 2023

Object rearrangement is a fundamental problem in robotics with various practical applications ranging from managing warehouses to cleaning and organizing home kitchens. While existing research has primarily focused on single-agent solutions, real-world scenarios often require multiple robots to work together on rearrangement tasks. This paper proposes a comprehensive learning-based framework for multi-agent object rearrangement planning, addressing the challenges of task sequencing and path planning in complex environments. The proposed method iteratively selects objects, determines their relocation regions, and pairs them with available robots under kinematic feasibility and task reachability for execution to achieve the target arrangement. Our experiments on a diverse range of simulated and real-world environments demonstrate the effectiveness and robustness of the proposed framework. Furthermore, results indicate improved performance in terms of traversal time and success rate compared to baseline approaches.
Local-to-global Perspectives on Graph Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

This thesis presents a local-to-global perspective on graph neural networks (GNN), the leading architecture to process graph-structured data. After categorizing GNN into local Message Passing Neural Networks (MPNN) and global Graph transformers, we present three pieces of work: 1) study the convergence property of a type of global GNN, Invariant Graph Networks, 2) connect the local MPNN and global Graph Transformer, and 3) use local MPNN for graph coarsening, a standard subroutine used in global modeling.
Inductive reasoning in humans and large language models
Venue: arXiv cs.CL 2023
Year: 2023

The impressive recent performance of large language models has led many to wonder to what extent they can serve as models of general intelligence or are similar to human cognition. We address this issue by applying GPT-3.5 and GPT-4 to a classic problem in human inductive reasoning known as property induction. Over two experiments, we elicit human judgments on a range of property induction tasks spanning multiple domains. Although GPT-3.5 struggles to capture many aspects of human behaviour, GPT-4 is much more successful: for the most part, its performance qualitatively matches that of humans, and the only notable exception is its failure to capture the phenomenon of premise non-monotonicity. Our work demonstrates that property induction allows for interesting comparisons between human and machine intelligence and provides two large datasets that can serve as benchmarks for future work in this vein.
Long-Term Autonomous Ocean Monitoring with Streaming Samples
Venue: arXiv cs.RO 2023
Year: 2023

In the autonomous ocean monitoring task, the sampling robot moves in the environment and accumulates data continuously. The widely adopted spatial modeling method - standard Gaussian process (GP) regression - becomes inadequate in processing the growing sensing data of a large size. To overcome the computational challenge, this paper presents an environmental modeling framework using a sparse variant of GP called streaming sparse GP (SSGP). The SSGP is able to handle streaming data in an online and incremental manner, and is therefore suitable for long-term autonomous environmental monitoring. The SSGP summarizes the collected data using a small set of pseudo data points that best represent the whole dataset, and updates the hyperparameters and pseudo point locations in a streaming fashion, leading to high-quality approximation of the underlying environmental model with significantly reduced computational cost and memory demand.
Unlocking the Power of Health Datasets and Registries: The Need for Urgent Institutional and National Ownership and Governance Regulations for Research Advancement
Venue: arXiv cs.CY 2023
Year: 2023

Health datasets have immense potential to drive research advancements and improve healthcare outcomes. However, realizing this potential requires careful consideration of governance and ownership frameworks. This article explores the importance of nurturing governance and ownership models that facilitate responsible and ethical use of health datasets for research purposes. We highlight the importance of adopting governance and ownership models that enable responsible and ethical utilization of health datasets and clinical data registries for research purposes. The article addresses the important local and international regulations related to the utilization of health data/medical records in research, and emphasizes the urgent need for developing clear institutional and national guidelines on data access, sharing, and utilization, ensuring transparency, privacy, and data protection. By establishing robust governance structures and fostering ownership among stakeholders, collaboration, innovation, and equitable access to health data can be promoted, ultimately unlocking its full power for transformative research and improving global health outcomes.
Blocked Cross-Validation: A Precise and Efficient Method for Hyperparameter Tuning
Venue: arXiv cs.LG 2023
Year: 2023

Hyperparameter tuning plays a crucial role in optimizing the performance of predictive learners. Cross--validation (CV) is a widely adopted technique for estimating the error of different hyperparameter settings. Repeated cross-validation (RCV) has been commonly employed to reduce the variability of CV errors. In this paper, we introduce a novel approach called blocked cross-validation (BCV), where the repetitions are blocked with respect to both CV partition and the random behavior of the learner. Theoretical analysis and empirical experiments demonstrate that BCV provides more precise error estimates compared to RCV, even with a significantly reduced number of runs. We present extensive examples using real--world data sets to showcase the effectiveness and efficiency of BCV in hyperparameter tuning. Our results indicate that BCV outperforms RCV in hyperparameter tuning, achieving greater precision with fewer computations.
Neural Projection Mapping Using Reflectance Fields
Venue: arXiv cs.CV 2023
Year: 2023

We introduce a high resolution spatially adaptive light source, or a projector, into a neural reflectance field that allows to both calibrate the projector and photo realistic light editing. The projected texture is fully differentiable with respect to all scene parameters, and can be optimized to yield a desired appearance suitable for applications in augmented reality and projection mapping. Our neural field consists of three neural networks, estimating geometry, material, and transmittance. Using an analytical BRDF model and carefully selected projection patterns, our acquisition process is simple and intuitive, featuring a fixed uncalibrated projected and a handheld camera with a co-located light source. As we demonstrate, the virtual projector incorporated into the pipeline improves scene understanding and enables various projection mapping applications, alleviating the need for time consuming calibration steps performed in a traditional setting per view or projector location. In addition to enabling novel viewpoint synthesis, we demonstrate state-of-the-art performance projector compensation for novel viewpoints, improvement over the baselines in material and scene reconstruction, and three simply implemented scenarios where projection image optimization is performed, including the use of a 2D generative model to consistently dictate scene appearance from multiple viewpoints. We believe that neural projection mapping opens up the door to novel and exciting downstream tasks, through the joint optimization of the scene and projection images.
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model
Venue: arXiv cs.CL 2023
Year: 2023

Currently, the reduction in the parameter scale of large-scale pre-trained language models (PLMs) through knowledge distillation has greatly facilitated their widespread deployment on various devices. However, the deployment of knowledge distillation systems faces great challenges in real-world industrial-strength applications, which require the use of complex distillation methods on even larger-scale PLMs (over 10B), limited by memory on GPUs and the switching of methods. To overcome these challenges, we propose GKD, a general knowledge distillation framework that supports distillation on larger-scale PLMs using various distillation methods. With GKD, developers can build larger distillation models on memory-limited GPUs and easily switch and combine different distillation methods within a single framework. Experimental results show that GKD can support the distillation of at least 100B-scale PLMs and 25 mainstream methods on 8 NVIDIA A100 (40GB) GPUs.
Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
Venue: arXiv cs.CV 2023
Year: 2023

We present Face0, a novel way to instantaneously condition a text-to-image generation model on a face, in sample time, without any optimization procedures such as fine-tuning or inversions. We augment a dataset of annotated images with embeddings of the included faces and train an image generation model, on the augmented dataset. Once trained, our system is practically identical at inference time to the underlying base model, and is therefore able to generate images, given a user-supplied face image and a prompt, in just a couple of seconds. Our method achieves pleasing results, is remarkably simple, extremely fast, and equips the underlying model with new capabilities, like controlling the generated images both via text or via direct manipulation of the input face embeddings. In addition, when using a fixed random vector instead of a face embedding from a user supplied image, our method essentially solves the problem of consistent character generation across images. Finally, while requiring further research, we hope that our method, which decouples the model's textual biases from its biases on faces, might be a step towards some mitigation of biases in future text-to-image models.
Detection and Recovery of Hidden Submatrices
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we study the problems of detection and recovery of hidden submatrices with elevated means inside a large Gaussian random matrix. We consider two different structures for the planted submatrices. In the first model, the planted matrices are disjoint, and their row and column indices can be arbitrary. Inspired by scientific applications, the second model restricts the row and column indices to be consecutive. In the detection problem, under the null hypothesis, the observed matrix is a realization of independent and identically distributed standard normal entries. Under the alternative, there exists a set of hidden submatrices with elevated means inside the same standard normal matrix. Recovery refers to the task of locating the hidden submatrices. For both problems, and for both models, we characterize the statistical and computational barriers by deriving information-theoretic lower bounds, designing and analyzing algorithms matching those bounds, and proving computational lower bounds based on the low-degree polynomials conjecture. In particular, we show that the space of the model parameters (i.e., number of planted submatrices, their dimensions, and elevated mean) can be partitioned into three regions: the impossible regime, where all algorithms fail; the hard regime, where while detection or recovery are statistically possible, we give some evidence that polynomial-time algorithm do not exist; and finally the easy regime, where polynomial-time algorithms exist.
Analysis of a contention-based approach over 5G NR for Federated Learning in an Industrial Internet of Things scenario
Venue: arXiv cs.NI 2023
Year: 2023

The growing interest in new applications involving co-located heterogeneous requirements, such as the Industrial Internet of Things (IIoT) paradigm, poses unprecedented challenges to the uplink wireless transmissions. Dedicated scheduling has been the fundamental approach used by mobile radio systems for uplink transmissions, where the network assigns contention-free resources to users based on buffer-related information. The usage of contention-based transmissions was discussed by the 3rd Generation Partnership Project (3GPP) as an alternative approach for reducing the uplink latency characterizing dedicated scheduling. Nevertheless, the contention-based approach was not considered for standardization in LTE due to limited performance gains. However, 5G NR introduced a different radio frame which could change the performance achievable with a contention-based framework, although this has not yet been evaluated. This paper aims to fill this gap. We present a contention-based design introduced for uplink transmissions in a 5G NR IIoT scenario. We provide an up-to-date analysis via near-product 3GPP-compliant network simulations of the achievable application-level performance with simultaneous Ultra-Reliable Low Latency Communications (URLLC) and Federated Learning (FL) traffic, where the contention-based scheme is applied to the FL traffic. The investigation also involves two separate mechanisms for handling retransmissions of lost or collided transmissions. Numerical results show that, under some conditions, the proposed contention-based design provides benefits over dedicated scheduling when considering FL upload/download times, and does not significantly degrade the performance of URLLC.
Efficient Skip Connections Realization for Secure Inference on Encrypted Data
Venue: arXiv cs.CR 2023
Year: 2023

Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for example, to perform secure classification. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. The latter appears to be very costly when attempting to execute model inference under HE. In this paper, we show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions, achieving x1.3 computing power improvement for the same accuracy.
Precise and Generalized Robustness Certification for Neural Networks
Venue: arXiv cs.CR 2023
Year: 2023

The objective of neural network (NN) robustness certification is to determine if a NN changes its predictions when mutations are made to its inputs. While most certification research studies pixel-level or a few geometrical-level and blurring operations over images, this paper proposes a novel framework, GCERT, which certifies NN robustness under a precise and unified form of diverse semantic-level image mutations. We formulate a comprehensive set of semantic-level image mutations uniformly as certain directions in the latent space of generative models. We identify two key properties, independence and continuity, that convert the latent space into a precise and analysis-friendly input space representation for certification. GCERT can be smoothly integrated with de facto complete, incomplete, or quantitative certification frameworks. With its precise input space representation, GCERT enables for the first time complete NN robustness certification with moderate cost under diverse semantic-level input mutations, such as weather-filter, style transfer, and perceptual changes (e.g., opening/closing eyes). We show that GCERT enables certifying NN robustness under various common and security-sensitive scenarios like autonomous driving.
Between-Sample Relationship in Learning Tabular Data Using Graph and Attention Networks
Venue: arXiv cs.LG 2023
Year: 2023

Traditional machine learning assumes samples in tabular data to be independent and identically distributed (i.i.d). This assumption may miss useful information within and between sample relationships in representation learning. This paper relaxes the i.i.d assumption to learn tabular data representations by incorporating between-sample relationships for the first time using graph neural networks (GNN). We investigate our hypothesis using several GNNs and state-of-the-art (SOTA) deep attention models to learn the between-sample relationship on ten tabular data sets by comparing them to traditional machine learning methods. GNN methods show the best performance on tabular data with large feature-to-sample ratios. Our results reveal that attention-based GNN methods outperform traditional machine learning on five data sets and SOTA deep tabular learning methods on three data sets. Between-sample learning via GNN and deep attention methods yield the best classification accuracy on seven of the ten data sets. This suggests that the i.i.d assumption may not always hold for most tabular data sets.
The capacity of quiver representations and the Anantharam-Jog-Nair inequality
Venue: arXiv cs.IT 2023
Year: 2023

The Anantharam-Jog-Nair inequality [AJN22] in Information Theory provides a unifying approach to the information-theoretic form of the Brascamp-Lieb inequality [CCE09] and the Entropy Power inequality [ZF93]. In this paper, we use methods from Quiver Invariant Theory [CD21] to study Anantharam-Jog-Nair inequalities with integral exponents. For such an inequality, we first view its input datum as a quiver datum and show that the best constant that occurs in the Anantharam-Jog-Nair inequality is of the form $-{1\over 2}\log (\mathbf{cap}(V,σ))$ where $\mathbf{cap}(V, σ)$ is the capacity of a quiver datum $(V, σ)$ of a complete bipartite quiver. The general tools developed in [CD21], when applied to complete bipartite quivers, yield necessary and sufficient conditions for: (1) the finiteness of the Anantharam-Jog-Nair best constants; and (2) the existence of Gaussian extremizers. These results recover some of the main results in [AJN22] and [ACZ22]. In addition, we characterize gaussian-extremizable data in terms of semi-simple data, and provide a general character formula for the Anatharam-Jog-Nair constants. Furthermore, our quiver invariant theoretic methods lead to necessary and sufficient conditions for the uniqueness of Gaussian extremizers. This answers the third and last question left unanswered in [AJN22].
On the Efficacy of 3D Point Cloud Reinforcement Learning
Venue: arXiv cs.RO 2023
Year: 2023

Recent studies on visual reinforcement learning (visual RL) have explored the use of 3D visual representations. However, none of these work has systematically compared the efficacy of 3D representations with 2D representations across different tasks, nor have they analyzed 3D representations from the perspective of agent-object / object-object relationship reasoning. In this work, we seek answers to the question of when and how do 3D neural networks that learn features in the 3D-native space provide a beneficial inductive bias for visual RL. We specifically focus on 3D point clouds, one of the most common forms of 3D representations. We systematically investigate design choices for 3D point cloud RL, leading to the development of a robust algorithm for various robotic manipulation and control tasks. Furthermore, through comparisons between 2D image vs 3D point cloud RL methods on both minimalist synthetic tasks and complex robotic manipulation tasks, we find that 3D point cloud RL can significantly outperform the 2D counterpart when agent-object / object-object relationship encoding is a key factor.
Designing Conversational Multimodal 3D Printed Models with People who are Blind
Venue: arXiv cs.HC 2023
Year: 2023

3D printed models have been used to improve access to graphical information by people who are blind, offering benefits over conventional accessible graphics. Here we investigate an interactive 3D printed model (I3M) that combines a conversational interface with haptic vibration and touch to provide more natural and accessible experiences. Specifically, we co-designed a multimodal model of the Solar System with nine blind people and evaluated the prototype with another seven blind participants. We discuss our journey from a design perspective, focusing on touch, conversational and multimodal interactions. Based on our experience, we suggest design recommendations that consider blind users' desire for independence and control, customisation, comfort and use of prior experience
Weakly supervised information extraction from inscrutable handwritten document images
Venue: arXiv cs.CV 2023
Year: 2023

State-of-the-art information extraction methods are limited by OCR errors. They work well for printed text in form-like documents, but unstructured, handwritten documents still remain a challenge. Adapting existing models to domain-specific training data is quite expensive, because of two factors, 1) limited availability of the domain-specific documents (such as handwritten prescriptions, lab notes, etc.), and 2) annotations become even more challenging as one needs domain-specific knowledge to decode inscrutable handwritten document images. In this work, we focus on the complex problem of extracting medicine names from handwritten prescriptions using only weakly labeled data. The data consists of images along with the list of medicine names in it, but not their location in the image. We solve the problem by first identifying the regions of interest, i.e., medicine lines from just weak labels and then injecting a domain-specific medicine language model learned using only synthetically generated data. Compared to off-the-shelf state-of-the-art methods, our approach performs >2.5x better in medicine names extraction from prescriptions.