OpenRaiser/Intern-Atlas huggingface dataset:
Emulating On-Orbit Interactions Using Forward Dynamics Based Cartesian Motion
Venue: arXiv cs.RO 2023
Year: 2023

On-orbit operations such as servicing and assembly are considered a priority for the future space industry. Ground-based facilities that emulate on-orbit interactions are key tools for developing and testing space technology. This paper presents a control framework to emulate on-orbit operations using on-ground robotic manipulators. It combines Virtual Forward Dynamics Models (VFDM) for Cartesian motion control of robotic manipulators with an Orbital Dynamics Simulator (ODS) based on the Clohessy Wiltshire (CW) Model. The VFDM-based Inverse Kinematics (IK) solver is known to have better motion tracking, path accuracy, and solver convergency than traditional IK solvers. Thus, it provides a stable Cartesian motion for manipulators based on orbit emulations, even at singular or near singular configurations. The framework is tested at the ZeroG-Lab robotic facility of the SnT by emulating two scenarios: free-floating satellite motion and free-floating interaction (collision). Results show fidelity between the simulated motion commanded by the ODS and the one executed by the robot-mounted mockups.
Holographic-(V)AE: an end-to-end SO(3)-Equivariant (Variational) Autoencoder in Fourier Space
Venue: arXiv cs.LG 2023
Year: 2023

Group-equivariant neural networks have emerged as a data-efficient approach to solve classification and regression tasks, while respecting the relevant symmetries of the data. However, little work has been done to extend this paradigm to the unsupervised and generative domains. Here, we present Holographic-(Variational) Auto Encoder (H-(V)AE), a fully end-to-end SO(3)-equivariant (variational) autoencoder in Fourier space, suitable for unsupervised learning and generation of data distributed around a specified origin in 3D. H-(V)AE is trained to reconstruct the spherical Fourier encoding of data, learning in the process a low-dimensional representation of the data (i.e., a latent space) with a maximally informative rotationally invariant embedding alongside an equivariant frame describing the orientation of the data. We extensively test the performance of H-(V)AE on diverse datasets. We show that the learned latent space efficiently encodes the categorical features of spherical images. Moreover, H-(V)AE's latent space can be used to extract compact embeddings for protein structure microenvironments, and when paired with a Random Forest Regressor, it enables state-of-the-art predictions of protein-ligand binding affinity.
Point normal orientation and surface reconstruction by incorporating isovalue constraints to Poisson equation
Venue: arXiv cs.GR 2023
Year: 2023

Oriented normals are common pre-requisites for many geometric algorithms based on point clouds, such as Poisson surface reconstruction. However, it is not trivial to obtain a consistent orientation. In this work, we bridge orientation and reconstruction in the implicit space and propose a novel approach to orient point cloud normals by incorporating isovalue constraints to the Poisson equation. In implicit surface reconstruction, the reconstructed shape is represented as an isosurface of an implicit function defined in the ambient space. Therefore, when such a surface is reconstructed from a set of sample points, the implicit function values at the points should be close to the isovalue corresponding to the surface. Based on this observation and the Poisson equation, we propose an optimization formulation that combines isovalue constraints with local consistency requirements for normals. We optimize normals and implicit functions simultaneously and solve for a globally consistent orientation. Thanks to the sparsity of the linear system, our method can work on an average laptop with reasonable computational time. Experiments show that our method can achieve high performance in non-uniform and noisy data and manage varying sampling densities, artifacts, multiple connected components, and nested surfaces. The source code is available at \url{https://github.com/Submanifold/IsoConstraints}.
An Efficient Modular Exponentiation Proof Scheme
Venue: arXiv cs.CR 2023
Year: 2023

We present an efficient proof scheme for any instance of left-to-right modular exponentiation, used in many computational tests for primality. Specifically, we show that for any $(a,n,r,m)$ the correctness of a computation $a^n\equiv r\pmod m$ can be proven and verified with an overhead negligible compared to the computational cost of the exponentiation. Our work generalizes the Gerbicz-Pietrzak proof scheme used when $n$ is a power of $2$, and has been successfully implemented at PrimeGrid, doubling the efficiency of distributed searches for primes.
Architecting materials for extremal stiffness, yield and buckling strength
Venue: arXiv cs.CE 2023
Year: 2023

This paper proposes a methodology for architecting microstructures with extremal stiffness, yield, and buckling strength using topology optimization. The optimized microstructures reveal an interesting transition from simple lattice like structures for yield-dominated situations to hierarchical lattice structures for buckling-dominated situations. The transition from simple to hierarchical is governed by the relative yield strength of the constituent base material as well as the volume fraction. The overall performances of the optimized microstructures indicate that maximum strength is determined by the buckling strength at low volume fractions and yield strength at higher volume fractions, regardless of the base material's relative yield strength. The non-normalized properties of the optimized microstructures show that higher base material Young's modulus leads to both higher Young's modulus and strength of the architected microstructures. Furthermore, the polynomial order of the maximum strength lines with respect to mass density obtained from the optimized microstructures reduces as base material relative yield strength decreases, reducing from 2.3 for buckling dominated Thermoplastic Polyurethane to 1 for yield dominated steel microstructures.
Joint Mobility Control and MEC Offloading for Hybrid Satellite-Terrestrial-Network-Enabled Robots
Venue: arXiv cs.IT 2023
Year: 2023

Benefiting from the fusion of communication and intelligent technologies, network-enabled robots have become important to support future machine-assisted and unmanned applications. To provide high-quality services for robots in wide areas, hybrid satellite-terrestrial networks are a key technology. Through hybrid networks, computation-intensive and latency-sensitive tasks can be offloaded to mobile edge computing (MEC) servers. However, due to the mobility of mobile robots and unreliable wireless network environments, excessive local computations and frequent service migrations may significantly increase the service delay. To address this issue, this paper aims to minimize the average task completion time for MEC-based offloading initiated by satellite-terrestrial-network-enabled robots. Different from conventional mobility-aware schemes, the proposed scheme makes the offloading decision by jointly considering the mobility control of robots. A joint optimization problem of task offloading and velocity control is formulated. Using Lyapunov optimization, the original optimization is decomposed into a velocity control subproblem and a task offloading subproblem. Then, based on the Markov decision process (MDP), a dual-agent reinforcement learning (RL) algorithm is proposed. The convergence and complexity of the improved RL algorithm are theoretically analyzed, and the simulation results show that the proposed scheme can effectively reduce the offloading delay.
Saving the Limping: Fault-tolerant Quadruped Locomotion via Reinforcement Learning
Venue: arXiv cs.RO 2023
Year: 2023

Modern quadrupeds are skillful in traversing or even sprinting on uneven terrains in a remote uncontrolled environment. However, survival in the wild requires not only maneuverability, but also the ability to handle potential critical hardware failures. How to grant such ability to quadrupeds is rarely investigated. In this paper, we propose a novel methodology to train and test hardware fault-tolerant controllers for quadruped locomotion, both in the simulation and physical world. We adopt the teacher-student reinforcement learning framework to train the controller with close-to-reality joint-locking failure in the simulation, which can be zero-shot transferred to the physical robot without any fine-tuning. Extensive experiments show that our fault-tolerant controller can efficiently lead a quadruped stably when it faces joint failures during locomotion.
Robust Empirical Risk Minimization with Tolerance
Venue: arXiv cs.LG 2023
Year: 2023

Developing simple, sample-efficient learning algorithms for robust classification is a pressing issue in today's tech-dominated world, and current theoretical techniques requiring exponential sample complexity and complicated improper learning rules fall far from answering the need. In this work we study the fundamental paradigm of (robust) $\textit{empirical risk minimization}$ (RERM), a simple process in which the learner outputs any hypothesis minimizing its training error. RERM famously fails to robustly learn VC classes (Montasser et al., 2019a), a bound we show extends even to `nice' settings such as (bounded) halfspaces. As such, we study a recent relaxation of the robust model called $\textit{tolerant}$ robust learning (Ashtiani et al., 2022) where the output classifier is compared to the best achievable error over slightly larger perturbation sets. We show that under geometric niceness conditions, a natural tolerant variant of RERM is indeed sufficient for $γ$-tolerant robust learning VC classes over $\mathbb{R}^d$, and requires only $\tilde{O}\left( \frac{VC(H)d\log \frac{D}{γδ}}{ε^2}\right)$ samples for robustness regions of (maximum) diameter $D$.
Relational Models for the Lambek Calculus with Intersection and Constants
Venue: arXiv cs.LO 2023
Year: 2023

We consider relational semantics (R-models) for the Lambek calculus extended with intersection and explicit constants for zero and unit. For its variant without constants and a restriction which disallows empty antecedents, Andreka and Mikulas (1994) prove strong completeness. We show that it fails without this restriction, but, on the other hand, prove weak completeness for non-standard interpretation of constants. For the standard interpretation, even weak completeness fails. The weak completeness result extends to an infinitary setting, for so-called iterative divisions (Kleene star under division). We also prove strong completeness results for product-free fragments.
Under the Cover Infant Pose Estimation using Multimodal Data
Venue: arXiv cs.CV 2023
Year: 2023

Infant pose monitoring during sleep has multiple applications in both healthcare and home settings. In a healthcare setting, pose detection can be used for region of interest detection and movement detection for noncontact based monitoring systems. In a home setting, pose detection can be used to detect sleep positions which has shown to have a strong influence on multiple health factors. However, pose monitoring during sleep is challenging due to heavy occlusions from blanket coverings and low lighting. To address this, we present a novel dataset, Simultaneously-collected multimodal Mannequin Lying pose (SMaL) dataset, for under the cover infant pose estimation. We collect depth and pressure imagery of an infant mannequin in different poses under various cover conditions. We successfully infer full body pose under the cover by training state-of-art pose estimation methods and leveraging existing multimodal adult pose datasets for transfer learning. We demonstrate a hierarchical pretraining strategy for transformer-based models to significantly improve performance on our dataset. Our best performing model was able to detect joints under the cover within 25mm 86% of the time with an overall mean error of 16.9mm. Data, code and models publicly available at https://github.com/DanielKyr/SMaL
Multi-view information fusion using multi-view variational autoencoders to predict proximal femoral strength
Venue: arXiv cs.LG 2023
Year: 2023

The aim of this paper is to design a deep learning-based model to predict proximal femoral strength using multi-view information fusion. Method: We developed new models using multi-view variational autoencoder (MVAE) for feature representation learning and a product of expert (PoE) model for multi-view information fusion. We applied the proposed models to an in-house Louisiana Osteoporosis Study (LOS) cohort with 931 male subjects, including 345 African Americans and 586 Caucasians. With an analytical solution of the product of Gaussian distribution, we adopted variational inference to train the designed MVAE-PoE model to perform common latent feature extraction. We performed genome-wide association studies (GWAS) to select 256 genetic variants with the lowest p-values for each proximal femoral strength and integrated whole genome sequence (WGS) features and DXA-derived imaging features to predict proximal femoral strength. Results: The best prediction model for fall fracture load was acquired by integrating WGS features and DXA-derived imaging features. The designed models achieved the mean absolute percentage error of 18.04%, 6.84% and 7.95% for predicting proximal femoral fracture loads using linear models of fall loading, nonlinear models of fall loading, and nonlinear models of stance loading, respectively. Compared to existing multi-view information fusion methods, the proposed MVAE-PoE achieved the best performance. Conclusion: The proposed models are capable of predicting proximal femoral strength using WGS features and DXA-derived imaging features. Though this tool is not a substitute for FEA using QCT images, it would make improved assessment of hip fracture risk more widely available while avoiding the increased radiation dosage and clinical costs from QCT.
Meta-Learning Priors for Safe Bayesian Optimization
Venue: arXiv cs.LG 2023
Year: 2023

In robotics, optimizing controller parameters under safety constraints is an important challenge. Safe Bayesian optimization (BO) quantifies uncertainty in the objective and constraints to safely guide exploration in such settings. Hand-designing a suitable probabilistic model can be challenging, however. In the presence of unknown safety constraints, it is crucial to choose reliable model hyper-parameters to avoid safety violations. Here, we propose a data-driven approach to this problem by meta-learning priors for safe BO from offline data. We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner via empirical uncertainty metrics and a frontier search algorithm. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches while maintaining safety.
Toward Trustworthy Neural Program Synthesis
Venue: arXiv cs.SE 2023
Year: 2023

We develop an approach to estimate the probability that a program sampled from a large language model is correct. Given a natural language description of a programming problem, our method samples both candidate programs as well as candidate predicates specifying how the program should behave. This allows learning a model that forms a well-calibrated probabilistic prediction of program correctness. Our system also infers which predicates are useful to explain the behavior of the generated code, and humans preferred these in a human study over raw language model outputs. Our method is simple, easy to implement, and maintains state of the art generation accuracy results.
Assuring Safety of Vision-Based Swarm Formation Control
Venue: arXiv cs.MA 2023
Year: 2023

Vision-based formation control systems are attractive because they can use inexpensive sensors and can work in GPS-denied environments. The safety assurance for such systems is challenging: the vision component's accuracy depends on the environment in complicated ways, these errors propagate through the system and lead to incorrect control actions, and there exists no formal specification for end-to-end reasoning. We address this problem and propose a technique for safety assurance of vision-based formation control: First, we propose a scheme for constructing quantizers that are consistent with vision-based perception. Next, we show how the convergence analysis of a standard quantized consensus algorithm can be adapted for the constructed quantizers. We use the recently defined notion of perception contracts to create error bounds on the actual vision-based perception pipeline using sampled data from different ground truth states, environments, and weather conditions. Specifically, we use a quantizer in logarithmic polar coordinates, and we show that this quantizer is suitable for the constructed perception contracts for the vision-based position estimation, where the error worsens with respect to the absolute distance between agents. We build our formation control algorithm with this nonuniform quantizer, and we prove its convergence employing an existing result for quantized consensus.
SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysis
Venue: arXiv cs.CL 2023
Year: 2023

We propose MINT, a new Multilingual INTimacy analysis dataset covering 13,372 tweets in 10 languages including English, French, Spanish, Italian, Portuguese, Korean, Dutch, Chinese, Hindi, and Arabic. We benchmarked a list of popular multilingual pre-trained language models. The dataset is released along with the SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysis (https://sites.google.com/umich.edu/semeval-2023-tweet-intimacy).
spred: Solving $L_1$ Penalty with SGD
Venue: arXiv cs.LG 2023
Year: 2023

We propose to minimize a generic differentiable objective with $L_1$ constraint using a simple reparametrization and straightforward stochastic gradient descent. Our proposal is the direct generalization of previous ideas that the $L_1$ penalty may be equivalent to a differentiable reparametrization with weight decay. We prove that the proposed method, \textit{spred}, is an exact differentiable solver of $L_1$ and that the reparametrization trick is completely ``benign" for a generic nonconvex function. Practically, we demonstrate the usefulness of the method in (1) training sparse neural networks to perform gene selection tasks, which involves finding relevant features in a very high dimensional space, and (2) neural network compression task, to which previous attempts at applying the $L_1$-penalty have been unsuccessful. Conceptually, our result bridges the gap between the sparsity in deep learning and conventional statistical learning.
One-shot Detail Retouching with Patch Space Neural Transformation Blending
Venue: arXiv cs.CV 2023
Year: 2023

Photo retouching is a difficult task for novice users as it requires expert knowledge and advanced tools. Photographers often spend a great deal of time generating high-quality retouched photos with intricate details. In this paper, we introduce a one-shot learning based technique to automatically retouch details of an input image based on just a single pair of before and after example images. Our approach provides accurate and generalizable detail edit transfer to new images. We achieve these by proposing a new representation for image to image maps. Specifically, we propose neural field based transformation blending in the patch space for defining patch to patch transformations for each frequency band. This parametrization of the map with anchor transformations and associated weights, and spatio-spectral localized patches, allows us to capture details well while staying generalizable. We evaluate our technique both on known ground truth filters and artist retouching edits. Our method accurately transfers complex detail retouching edits.
Cost Design in Atomic Routing Games
Venue: arXiv cs.GT 2023
Year: 2023

An atomic routing game is a multiplayer game on a directed graph. Each player in the game chooses a path -- a sequence of links that connect its origin node to its destination node -- with the lowest cost, where the cost of each link is a function of all players' choices. We develop a novel numerical method to design the link cost function in atomic routing games such that the players' choices at the Nash equilibrium minimize a given smooth performance function. This method first approximates the nonsmooth Nash equilibrium conditions with smooth ones, then iteratively improves the link cost function via implicit differentiation. We demonstrate the application of this method to atomic routing games that model noncooperative agents navigating in grid worlds.
Improved High-Probability Regret for Adversarial Bandits with Time-Varying Feedback Graphs
Venue: arXiv cs.LG 2023
Year: 2023

We study high-probability regret bounds for adversarial $K$-armed bandits with time-varying feedback graphs over $T$ rounds. For general strongly observable graphs, we develop an algorithm that achieves the optimal regret $\widetilde{\mathcal{O}}((\sum_{t=1}^Tα_t)^{1/2}+\max_{t\in[T]}α_t)$ with high probability, where $α_t$ is the independence number of the feedback graph at round $t$. Compared to the best existing result [Neu, 2015] which only considers graphs with self-loops for all nodes, our result not only holds more generally, but importantly also removes any $\text{poly}(K)$ dependence that can be prohibitively large for applications such as contextual bandits. Furthermore, we also develop the first algorithm that achieves the optimal high-probability regret bound for weakly observable graphs, which even improves the best expected regret bound of [Alon et al., 2015] by removing the $\mathcal{O}(\sqrt{KT})$ term with a refined analysis. Our algorithms are based on the online mirror descent framework, but importantly with an innovative combination of several techniques. Notably, while earlier works use optimistic biased loss estimators for achieving high-probability bounds, we find it important to use a pessimistic one for nodes without self-loop in a strongly observable graph.
A Novel Light Field Coding Scheme Based on Deep Belief Network & Weighted Binary Images for Additive Layered Displays
Venue: arXiv cs.CV 2023
Year: 2023

Light-field displays create an immersive experience by providing binocular depth sensation and motion parallax. Stacking light attenuating layers is one approach to implement a light field display with a broader depth of field, wide viewing angles and high resolution. Due to the transparent holographic optical element (HOE) layers, additive layered displays can be integrated into augmented reality (AR) wearables to overlay virtual objects onto the real world, creating a seamless mixed reality (XR) experience. This paper proposes a novel framework for light field representation and coding that utilizes Deep Belief Network (DBN) and weighted binary images suitable for additive layered displays. The weighted binary representation of layers makes the framework more flexible for adaptive bitrate encoding. The framework effectively captures intrinsic redundancies in the light field data, and thus provides a scalable solution for light field coding suitable for XR display applications. The latent code is encoded by H.265 codec generating a rate-scalable bit-stream. We achieve adaptive bitrate decoding by varying the number of weighted binary images and the H.265 quantization parameter, while maintaining an optimal reconstruction quality. The framework is tested on real and synthetic benchmark datasets, and the results validate the rate-scalable property of the proposed scheme.