OpenRaiser/Intern-Atlas huggingface dataset:
StereoTac: a Novel Visuotactile Sensor that Combines Tactile Sensing with 3D Vision
Venue: arXiv cs.RO 2023
Year: 2023

Combining 3D vision with tactile sensing could unlock a greater level of dexterity for robots and improve several manipulation tasks. However, obtaining a close-up 3D view of the location where manipulation contacts occur can be challenging, particularly in confined spaces, cluttered environments, or without installing more sensors on the end effector. In this context, this paper presents StereoTac, a novel vision-based sensor that combines tactile sensing with 3D vision. The proposed sensor relies on stereoscopic vision to capture a 3D representation of the environment before contact and uses photometric stereo to reconstruct the tactile imprint generated by an object during contact. To this end, two cameras were integrated in a single sensor, whose interface is made of a transparent elastomer coated with a thin layer of paint with a level of transparency that can be adjusted by varying the sensor's internal lighting conditions. We describe the sensor's fabrication and evaluate its performance for both tactile perception and 3D vision. Our results show that the proposed sensor can reconstruct a 3D view of a scene just before grasping and perceive the tactile imprint after grasping, allowing for monitoring of the contact during manipulation.
Towards Universal Vision-language Omni-supervised Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Existing open-world universal segmentation approaches usually leverage CLIP and pre-computed proposal masks to treat open-world segmentation tasks as proposal classification. However, 1) these works cannot handle universal segmentation in an end-to-end manner, and 2) the limited scale of panoptic datasets restricts the open-world segmentation ability on things classes. In this paper, we present Vision-Language Omni-Supervised Segmentation (VLOSS). VLOSS starts from a Mask2Former universal segmentation framework with CLIP text encoder. To improve the open-world segmentation ability, we leverage omni-supervised data (i.e., panoptic segmentation data, object detection data, and image-text pairs data) into training, thus enriching the open-world segmentation ability and achieving better segmentation accuracy. To better improve the training efficiency and fully release the power of omni-supervised data, we propose several advanced techniques, i.e., FPN-style encoder, switchable training technique, and positive classification loss. Benefiting from the end-to-end training manner with proposed techniques, VLOSS can be applied to various open-world segmentation tasks without further adaptation. Experimental results on different open-world panoptic and instance segmentation benchmarks demonstrate the effectiveness of VLOSS. Notably, with fewer parameters, our VLOSS with Swin-Tiny backbone surpasses MaskCLIP by ~2% in terms of mask AP on LVIS v1 dataset.
Predicting Hurricane Evacuation Decisions with Interpretable Machine Learning Models
Venue: arXiv cs.LG 2023
Year: 2023

The aggravating effects of climate change and the growing population in hurricane-prone areas escalate the challenges in large-scale hurricane evacuations. While hurricane preparedness and response strategies vastly rely on the accuracy and timeliness of the predicted households' evacuation decisions, current studies featuring psychological-driven linear models leave some significant limitations in practice. Hence, the present study proposes a new methodology for predicting households' evacuation decisions constructed by easily accessible demographic and resource-related predictors compared to current models with a high reliance on psychological factors. Meanwhile, an enhanced logistic regression (ELR) model that could automatically account for nonlinearities (i.e., univariate and bivariate threshold effects) by an interpretable machine learning approach is developed to secure the accuracy of the results. Specifically, low-depth decision trees are selected for nonlinearity detection to identify the critical thresholds, build a transparent model structure, and solidify the robustness. Then, an empirical dataset collected after Hurricanes Katrina and Rita is hired to examine the practicability of the new methodology. The results indicate that the enhanced logistic regression (ELR) model has the most convincing performance in explaining the variation of the households' evacuation decision in model fit and prediction capability compared to previous linear models. It suggests that the proposed methodology could provide a new tool and framework for the emergency management authorities to improve the estimation of evacuation traffic demands in a timely and accurate manner.
Improve Retrieval-based Dialogue System via Syntax-Informed Attention
Venue: arXiv cs.AI 2023
Year: 2023

Multi-turn response selection is a challenging task due to its high demands on efficient extraction of the matching features from abundant information provided by context utterances. Since incorporating syntactic information like dependency structures into neural models can promote a better understanding of the sentences, such a method has been widely used in NLP tasks. Though syntactic information helps models achieved pleasing results, its application in retrieval-based dialogue systems has not been fully explored. Meanwhile, previous works focus on intra-sentence syntax alone, which is far from satisfactory for the task of multi-turn response where dialogues usually contain multiple sentences. To this end, we propose SIA, Syntax-Informed Attention, considering both intra- and inter-sentence syntax information. While the former restricts attention scope to only between tokens and corresponding dependents in the syntax tree, the latter allows attention in cross-utterance pairs for those syntactically important tokens. We evaluate our method on three widely used benchmarks and experimental results demonstrate the general superiority of our method on dialogue response selection.
MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation
Venue: arXiv cs.CL 2023
Year: 2023

Recent approaches to word sense disambiguation (WSD) utilize encodings of the sense gloss (definition), in addition to the input context, to improve performance. In this work we demonstrate that this approach can be adapted for use in multiword expression (MWE) identification by training models which use gloss and context information to filter MWE candidates produced by a rule-based extraction pipeline. Our approach substantially improves precision, outperforming the state-of-the-art in MWE identification on the DiMSUM dataset by up to 1.9 F1 points and achieving competitive results on the PARSEME 1.1 English dataset. Our models also retain most of their WSD performance, showing that a single model can be used for both tasks. Finally, building on similar approaches using Bi-encoders for WSD, we introduce a novel Poly-encoder architecture which improves MWE identification performance.
From Compass and Ruler to Convolution and Nonlinearity: On the Surprising Difficulty of Understanding a Simple CNN Solving a Simple Geometric Estimation Task
Venue: arXiv cs.LG 2023
Year: 2023

Neural networks are omnipresent, but remain poorly understood. Their increasing complexity and use in critical systems raises the important challenge to full interpretability. We propose to address a simple well-posed learning problem: estimating the radius of a centred pulse in a one-dimensional signal or of a centred disk in two-dimensional images using a simple convolutional neural network. Surprisingly, understanding what trained networks have learned is difficult and, to some extent, counter-intuitive. However, an in-depth theoretical analysis in the one-dimensional case allows us to comprehend constraints due to the chosen architecture, the role of each filter and of the nonlinear activation function, and every single value taken by the weights of the model. Two fundamental concepts of neural networks arise: the importance of invariance and of the shape of the nonlinear activation functions.
Twice Regularized Markov Decision Processes: The Equivalence between Robustness and Regularization
Venue: arXiv cs.LG 2023
Year: 2023

Robust Markov decision processes (MDPs) aim to handle changing or partially known system dynamics. To solve them, one typically resorts to robust optimization methods. However, this significantly increases computational complexity and limits scalability in both learning and planning. On the other hand, regularized MDPs show more stability in policy learning without impairing time complexity. Yet, they generally do not encompass uncertainty in the model dynamics. In this work, we aim to learn robust MDPs using regularization. We first show that regularized MDPs are a particular instance of robust MDPs with uncertain reward. We thus establish that policy iteration on reward-robust MDPs can have the same time complexity as on regularized MDPs. We further extend this relationship to MDPs with uncertain transitions: this leads to a regularization term with an additional dependence on the value function. We then generalize regularized MDPs to twice regularized MDPs ($\text{R}^2$ MDPs), i.e., MDPs with $\textit{both}$ value and policy regularization. The corresponding Bellman operators enable us to derive planning and learning schemes with convergence and generalization guarantees, thus reducing robustness to regularization. We numerically show this two-fold advantage on tabular and physical domains, highlighting the fact that $\text{R}^2$ preserves its efficacy in continuous environments.
MizAR 60 for Mizar 50
Venue: arXiv cs.AI 2023
Year: 2023

As a present to Mizar on its 50th anniversary, we develop an AI/TP system that automatically proves about 60\% of the Mizar theorems in the hammer setting. We also automatically prove 75\% of the Mizar theorems when the automated provers are helped by using only the premises used in the human-written Mizar proofs. We describe the methods and large-scale experiments leading to these results. This includes in particular the E and Vampire provers, their ENIGMA and Deepire learning modifications, a number of learning-based premise selection methods, and the incremental loop that interleaves growing a corpus of millions of ATP proofs with training increasingly strong AI/TP systems on them. We also present a selection of Mizar problems that were proved automatically.
Non-Trivial Query Sampling For Efficient Learning To Plan
Venue: arXiv cs.RO 2023
Year: 2023

In recent years, learning-based approaches have revolutionized motion planning. The data generation process for these methods involves caching a large number of high quality paths for different queries (start, goal pairs) in various environments. Conventionally, a uniform random strategy is used for sampling these queries. However, this leads to inclusion of "trivial paths" in the dataset (e.g.,, straight line paths in case of length-optimal planning), which can be solved efficiently if the planner has access to a steering function. This work proposes a "non-trivial" query sampling procedure to add more complex paths in the dataset. Numerical experiments show that a higher success rate can be attained for neural planners trained on such a non-trivial dataset.
Transformation-Invariant Network for Few-Shot Object Detection in Remote Sensing Images
Venue: arXiv cs.CV 2023
Year: 2023

Object detection in remote sensing images relies on a large amount of labeled data for training. However, the increasing number of new categories and class imbalance make exhaustive annotation impractical. Few-shot object detection (FSOD) addresses this issue by leveraging meta-learning on seen base classes and fine-tuning on novel classes with limited labeled samples. Nonetheless, the substantial scale and orientation variations of objects in remote sensing images pose significant challenges to existing few-shot object detection methods. To overcome these challenges, we propose integrating a feature pyramid network and utilizing prototype features to enhance query features, thereby improving existing FSOD methods. We refer to this modified FSOD approach as a Strong Baseline, which has demonstrated significant performance improvements compared to the original baselines. Furthermore, we tackle the issue of spatial misalignment caused by orientation variations between the query and support images by introducing a Transformation-Invariant Network (TINet). TINet ensures geometric invariance and explicitly aligns the features of the query and support branches, resulting in additional performance gains while maintaining the same inference speed as the Strong Baseline. Extensive experiments on three widely used remote sensing object detection datasets, i.e., NWPU VHR-10.v2, DIOR, and HRRSD demonstrated the effectiveness of the proposed method.
Why do Tweeters regret sharing? Impacts of Twitter users' perception of sharing risk, perceived problems on Twitter, and the motivation of use on their behavior of regret sharing
Venue: arXiv cs.SI 2023
Year: 2023

This study presents a secondary data analysis of the survey data collected as part of the American Trends Panel series by the Pew Research Center. A logistic regression was performed to ascertain the effects of the perceived risk of sharing, perceived problems on Twitter, and motivation of using Twitter on the likelihood that participants regret sharing on Twitter. The logistic regression model was statistically significant, \c{hi}2(15) = 102.5, p < .001. The model correctly classified 78.5 percent of cases. Whether or not Twitter users regret sharing on Twitter depends on different motivations for using Twitter. We observe that "A way to express my opinion" is statistically significant in the mod-el, indicating that the odds of Twitter users regretting sharing for this motivation is 2.1 times higher than that of entertainment. Perceived risks of potential hostility and visibility were negatively associated with an increased likelihood of regret sharing. In contrast, perceived problems on Twitter concerning misinformation were negatively associated with the likelihood of regret sharing.
Four infinite families of ternary cyclic codes with a square-root-like lower bound
Venue: arXiv cs.IT 2023
Year: 2023

Cyclic codes are an interesting type of linear codes and have wide applications in communication and storage systems due to their efficient encoding and decoding algorithms. Inspired by the recent work on binary cyclic codes published in IEEE Trans. Inf. Theory, vol. 68, no. 12, pp. 7842-7849, 2022, and the arXiv paper arXiv:2301.06446, the objectives of this paper are the construction and analyses of four infinite families of ternary cyclic codes with length $n=3^m-1$ for odd $m$ and dimension $k \in \{n/2, (n + 2)/2\}$ whose minimum distances have a square-root-like lower bound. Their duals have parameters $[n, k^\perp, d^\perp]$, where $k^\perp \in \{n/2, (n- 2)/2\}$ and $d^\perp$ also has a square-root-like lower bound. These families of codes and their duals contain distance-optimal cyclic codes.
Optimal and Heuristic Min-Reg Scheduling Algorithms for GPU Programs
Venue: arXiv cs.PL 2023
Year: 2023

Given a basic block of instructions, finding a schedule that requires the minimum number of registers for evaluation is a well-known problem. The problem is NP-complete when the dependences among instructions form a directed-acyclic graph instead of a tree. We are striving to find efficient approximation algorithms for this problem not simply because it is an interesting graph optimization problem in theory. A good solution to this problem is also an essential component in solving the more complex instruction scheduling problem on GPU. In this paper, we start with explanations on why this problem is important in GPU instruction scheduling. We then explore two different approaches to tackling this problem. First we model this problem as a constraint-programming problem. Using a state-of-the-art CP-SAT solver, we can find optimal answers for much larger cases than previous works on a modest desktop PC. Second, guided by the optimal answers, we design and evaluate heuristics that can be applied to the polynomial-time list scheduling algorithms. A combination of those heuristics can achieve the register-pressure results that are about 17\% higher than the optimal minimum on average. However, there are still near 6\% cases in which the register pressure by the heuristic approach is 50\% higher than the optimal minimum.
Physics-driven machine learning models coupling PyTorch and Firedrake
Venue: arXiv cs.LG 2023
Year: 2023

Partial differential equations (PDEs) are central to describing and modelling complex physical systems that arise in many disciplines across science and engineering. However, in many realistic applications PDE modelling provides an incomplete description of the physics of interest. PDE-based machine learning techniques are designed to address this limitation. In this approach, the PDE is used as an inductive bias enabling the coupled model to rely on fundamental physical laws while requiring less training data. The deployment of high-performance simulations coupling PDEs and machine learning to complex problems necessitates the composition of capabilities provided by machine learning and PDE-based frameworks. We present a simple yet effective coupling between the machine learning framework PyTorch and the PDE system Firedrake that provides researchers, engineers and domain specialists with a high productive way of specifying coupled models while only requiring trivial changes to existing code.
Synthesizing Realistic Image Restoration Training Pairs: A Diffusion Approach
Venue: arXiv cs.CV 2023
Year: 2023

In supervised image restoration tasks, one key issue is how to obtain the aligned high-quality (HQ) and low-quality (LQ) training image pairs. Unfortunately, such HQ-LQ training pairs are hard to capture in practice, and hard to synthesize due to the complex unknown degradation in the wild. While several sophisticated degradation models have been manually designed to synthesize LQ images from their HQ counterparts, the distribution gap between the synthesized and real-world LQ images remains large. We propose a new approach to synthesizing realistic image restoration training pairs using the emerging denoising diffusion probabilistic model (DDPM). First, we train a DDPM, which could convert a noisy input into the desired LQ image, with a large amount of collected LQ images, which define the target data distribution. Then, for a given HQ image, we synthesize an initial LQ image by using an off-the-shelf degradation model, and iteratively add proper Gaussian noises to it. Finally, we denoise the noisy LQ image using the pre-trained DDPM to obtain the final LQ image, which falls into the target distribution of real-world LQ images. Thanks to the strong capability of DDPM in distribution approximation, the synthesized HQ-LQ image pairs can be used to train robust models for real-world image restoration tasks, such as blind face image restoration and blind image super-resolution. Experiments demonstrated the superiority of our proposed approach to existing degradation models. Code and data will be released.
Periodic handover skipping in cellular networks: Spatially stochastic modeling and analysis
Venue: arXiv cs.IT 2023
Year: 2023

Handover (HO) management is one of the most crucial tasks in dense cellular networks with mobile users. A problem in the HO management is to deal with increasing HOs due to network densification in the 5G evolution and various HO skipping techniques have so far been studied in the literature to suppress excessive HOs. In this paper, we propose yet another HO skipping scheme, called periodic HO skipping. The proposed scheme prohibits the HOs of a mobile user equipment (UE) for a certain period of time, referred to as skipping period, thereby enabling flexible operation of the HO skipping by adjusting the length of the skipping period. We investigate the performance of the proposed scheme on the basis of stochastic geometry. Specifically, we derive analytical expressions of two performance metrics -- the HO rate and the expected downlink data rate -- when a UE adopts the periodic HO skipping. Numerical results based on the analysis demonstrate that the periodic HO skipping scenario can outperform the scenario without any HO skipping in terms of a certain utility metric representing the trade-off between the HO rate and the expected downlink data rate, in particular when the UE moves fast. Furthermore, we numerically show that there can exist an optimal length of the skipping period, which locally maximizes the utility metric, and approximately provide the optimal skipping period in a simple form. Numerical comparison with some other HO skipping techniques is also conducted.
$\nabla$SD: Differentiable Programming for Sparse Tensors
Venue: arXiv cs.PL 2023
Year: 2023

Sparse tensors are prevalent in many data-intensive applications, yet existing differentiable programming frameworks are tailored towards dense tensors. This presents a significant challenge for efficiently computing gradients through sparse tensor operations, as their irregular sparsity patterns can result in substantial memory and computational overheads. In this work, we introduce a novel framework that enables the efficient and automatic differentiation of sparse tensors, addressing this fundamental issue. Our experiments demonstrate the effectiveness of the proposed framework in terms of performance and scalability, outperforming state-of-the-art frameworks across a range of synthetic and real-world datasets. Our approach offers a promising direction for enabling efficient and scalable differentiable programming with sparse tensors, which has significant implications for numerous applications in machine learning, natural language processing, and scientific computing.
Time-Optimal Path Tracking for Cooperative Manipulators: A Convex Optimization Approach
Venue: arXiv cs.RO 2023
Year: 2023

This paper studies the time-optimal path tracking problem for a team of cooperating robotic manipulators carrying an object. Considering the problem for rigidly grasped objects, we show that it can be cast as a convex optimization problem and solved efficiently with a guarantee of optimality. When formulating the problem, we avoid using a particular wrench distribution and exploit the full actuation available to the system. Then, we consider the problem for grasps using frictional forces and show that this problem also, under a force-closure grasp assumption, can be formulated as a convex optimization problem and solved efficiently and to optimality. To ensure a firm grasp, internal forces have been taken into account in this approach.
Uncertainty quantification in neural network classifiers -- a local linear approach
Venue: arXiv cs.LG 2023
Year: 2023

Classifiers based on neural networks (NN) often lack a measure of uncertainty in the predicted class. We propose a method to estimate the probability mass function (PMF) of the different classes, as well as the covariance of the estimated PMF. First, a local linear approach is used during the training phase to recursively compute the covariance of the parameters in the NN. Secondly, in the classification phase another local linear approach is used to propagate the covariance of the learned NN parameters to the uncertainty in the output of the last layer of the NN. This allows for an efficient Monte Carlo (MC) approach for: (i) estimating the PMF; (ii) calculating the covariance of the estimated PMF; and (iii) proper risk assessment and fusion of multiple classifiers. Two classical image classification tasks, i.e., MNIST, and CFAR10, are used to demonstrate the efficiency the proposed method.