OpenRaiser/Intern-Atlas huggingface dataset:
Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization
Venue: arXiv cs.RO 2024
Year: 2024

Generalizable manipulation requires that robots be able to interact with novel objects and environment. This requirement makes manipulation extremely challenging as a robot has to reason about complex frictional interactions with uncertainty in physical properties of the object and the environment. In this paper, we study robust optimization for planning of pivoting manipulation in the presence of uncertainties. We present insights about how friction can be exploited to compensate for inaccuracies in the estimates of the physical properties during manipulation. Under certain assumptions, we derive analytical expressions for stability margin provided by friction during pivoting manipulation. This margin is then used in a Contact Implicit Bilevel Optimization (CIBO) framework to optimize a trajectory that maximizes this stability margin to provide robustness against uncertainty in several physical parameters of the object. We present analysis of the stability margin with respect to several parameters involved in the underlying bilevel optimization problem. We demonstrate our proposed method using a 6 DoF manipulator for manipulating several different objects. We also design and validate an MPC controller using the proposed algorithm which can track and regulate the position of the object during manipulation.
A Generative Model for Digital Camera Noise Synthesis
Venue: arXiv cs.CV 2024
Year: 2024

Noise synthesis is a challenging low-level vision task aiming to generate realistic noise given a clean image along with the camera settings. To this end, we propose an effective generative model which utilizes clean features as guidance followed by noise injections into the network. Specifically, our generator follows a UNet-like structure with skip connections but without downsampling and upsampling layers. Firstly, we extract deep features from a clean image as the guidance and concatenate a Gaussian noise map to the transition point between the encoder and decoder as the noise source. Secondly, we propose noise synthesis blocks in the decoder in each of which we inject Gaussian noise to model the noise characteristics. Thirdly, we propose to utilize an additional Style Loss and demonstrate that this allows better noise characteristics supervision in the generator. Through a number of new experiments, we evaluate the temporal variance and the spatial correlation of the generated noise which we hope can provide meaningful insights for future works. Finally, we show that our proposed approach outperforms existing methods for synthesizing camera noise.
Learning-Based Modeling of Human-Autonomous Vehicle Interaction for Improved Safety in Mixed-Vehicle Platooning Control
Venue: arXiv cs.RO 2024
Year: 2024

The rising presence of autonomous vehicles (AVs) on public roads necessitates the development of advanced control strategies that account for the unpredictable nature of human-driven vehicles (HVs). This study introduces a learning-based method for modeling HV behavior, combining a traditional first-principles approach with a Gaussian process (GP) learning component. This hybrid model enhances the accuracy of velocity predictions and provides measurable uncertainty estimates. We leverage this model to develop a GP-based model predictive control (GP-MPC) strategy to improve safety in mixed vehicle platoons by integrating uncertainty assessments into distance constraints. Comparative simulations between our GP-MPC approach and a conventional model predictive control (MPC) strategy reveal that the GP-MPC ensures safer distancing and more efficient travel within the mixed platoon. By incorporating sparse GP modeling for HVs and a dynamic GP prediction in MPC, we significantly reduce the computation time of GP-MPC, making it only marginally longer than standard MPC and approximately 100 times faster than previous models not employing these techniques. Our findings underscore the effectiveness of learning-based HV modeling in enhancing safety and efficiency in mixed-traffic environments involving AV and HV interactions.
Efficient Diffusion Training via Min-SNR Weighting Strategy
Venue: arXiv cs.CV 2024
Year: 2024

Denoising diffusion models have been a mainstream approach for image generation, however, training these models often suffers from slow convergence. In this paper, we discovered that the slow convergence is partly due to conflicting optimization directions between timesteps. To address this issue, we treat the diffusion training as a multi-task learning problem, and introduce a simple yet effective approach referred to as Min-SNR-$γ$. This method adapts loss weights of timesteps based on clamped signal-to-noise ratios, which effectively balances the conflicts among timesteps. Our results demonstrate a significant improvement in converging speed, 3.4$\times$ faster than previous weighting strategies. It is also more effective, achieving a new record FID score of 2.06 on the ImageNet $256\times256$ benchmark using smaller architectures than that employed in previous state-of-the-art. The code is available at https://github.com/TiankaiHang/Min-SNR-Diffusion-Training.
SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolution
Venue: arXiv cs.CV 2024
Year: 2024

Previous works have shown that increasing the window size for Transformer-based image super-resolution models (e.g., SwinIR) can significantly improve the model performance. Still, the computation overhead is also considerable when the window size gradually increases. In this paper, we present SRFormer, a simple but novel method that can enjoy the benefit of large window self-attention but introduces even less computational burden. The core of our SRFormer is the permuted self-attention (PSA), which strikes an appropriate balance between the channel and spatial information for self-attention. Without any bells and whistles, we show that our SRFormer achieves a 33.86dB PSNR score on the Urban100 dataset, which is 0.46dB higher than that of SwinIR but uses fewer parameters and computations. In addition, we also attempt to scale up the model by further enlarging the window size and channel numbers to explore the potential of Transformer-based models. Experiments show that our scaled model, named SRFormerV2, can further improve the results and achieves state-of-the-art. We hope our simple and effective approach could be useful for future research in super-resolution model design. The homepage is https://z-yupeng.github.io/SRFormer/.
Inferring Occluded Agent Behavior in Dynamic Games from Noise Corrupted Observations
Venue: arXiv cs.MA 2024
Year: 2024

In mobile robotics and autonomous driving, it is natural to model agent interactions as the Nash equilibrium of a noncooperative, dynamic game. These methods inherently rely on observations from sensors such as lidars and cameras to identify agents participating in the game and, therefore, have difficulty when some agents are occluded. To address this limitation, this paper presents an occlusion-aware game-theoretic inference method to estimate the locations of potentially occluded agents, and simultaneously infer the intentions of both visible and occluded agents, which best accounts for the observations of visible agents. Additionally, we propose a receding horizon planning strategy based on an occlusion-aware contingency game designed to navigate in scenarios with potentially occluded agents. Monte Carlo simulations validate our approach, demonstrating that it accurately estimates the game model and trajectories for both visible and occluded agents using noisy observations of visible agents. Our planning pipeline significantly enhances navigation safety when compared to occlusion-ignorant baseline as well.
SpiRobs: Logarithmic Spiral-shaped Robots for Versatile Grasping Across Scales
Venue: arXiv cs.RO 2024
Year: 2024

Realizing a soft manipulator with biologically comparable flexibility and versatility often requires careful selection of materials and actuation, as well as attentive design of its structure, perception, and control. Here, we report a new class of soft robots (SpiRobs) that morphologically replicates the logarithmic spiral pattern observed in natural appendages (e.g., octopus arms, elephant trunks, etc.). This allows for a common design principle across different scales and a speedy and inexpensive fabrication process. We further present a grasping strategy inspired by the octopus that can automatically adapt to a target object's size and shape. We illustrate the dexterity of SpiRobs and the ability to tightly grasp objects that vary in size by more than two orders of magnitude and up to 260 times self-weight. We demonstrate scalability via three additional variants: a miniaturized gripper (mm), a one-meter-long manipulator, and an array of SpiRobs that can tangle up various objects.
Alternate Loss Functions for Classification and Robust Regression Can Improve the Accuracy of Artificial Neural Networks
Venue: arXiv cs.NE 2024
Year: 2024

All machine learning algorithms use a loss, cost, utility or reward function to encode the learning objective and oversee the learning process. This function that supervises learning is a frequently unrecognized hyperparameter that determines how incorrect outputs are penalized and can be tuned to improve performance. This paper shows that training speed and final accuracy of neural networks can significantly depend on the loss function used to train neural networks. In particular derivative values can be significantly different with different loss functions leading to significantly different performance after gradient descent based Backpropagation (BP) training. This paper explores the effect on performance of using new loss functions that are also convex but penalize errors differently compared to the popular Cross-entropy loss. Two new classification loss functions that significantly improve performance on a wide variety of benchmark tasks are proposed. A new loss function call smooth absolute error that outperforms the Squared error, Huber and Log-Cosh losses on datasets with significantly many outliers is proposed. This smooth absolute error loss function is infinitely differentiable and more closely approximates the absolute error loss compared to the Huber and Log-Cosh losses used for robust regression.
Optimal Horizon-Free Reward-Free Exploration for Linear Mixture MDPs
Venue: arXiv cs.LG 2024
Year: 2024

We study reward-free reinforcement learning (RL) with linear function approximation, where the agent works in two phases: (1) in the exploration phase, the agent interacts with the environment but cannot access the reward; and (2) in the planning phase, the agent is given a reward function and is expected to find a near-optimal policy based on samples collected in the exploration phase. The sample complexities of existing reward-free algorithms have a polynomial dependence on the planning horizon, which makes them intractable for long planning horizon RL problems. In this paper, we propose a new reward-free algorithm for learning linear mixture Markov decision processes (MDPs), where the transition probability can be parameterized as a linear combination of known feature mappings. At the core of our algorithm is uncertainty-weighted value-targeted regression with exploration-driven pseudo-reward and a high-order moment estimator for the aleatoric and epistemic uncertainties. When the total reward is bounded by $1$, we show that our algorithm only needs to explore $\tilde O( d^2\varepsilon^{-2})$ episodes to find an $\varepsilon$-optimal policy, where $d$ is the dimension of the feature mapping. The sample complexity of our algorithm only has a polylogarithmic dependence on the planning horizon and therefore is "horizon-free". In addition, we provide an $Ω(d^2\varepsilon^{-2})$ sample complexity lower bound, which matches the sample complexity of our algorithm up to logarithmic factors, suggesting that our algorithm is optimal.
Operating critical machine learning models in resource constrained regimes
Venue: arXiv cs.LG 2024
Year: 2024

The accelerated development of machine learning methods, primarily deep learning, are causal to the recent breakthroughs in medical image analysis and computer aided intervention. The resource consumption of deep learning models in terms of amount of training data, compute and energy costs are known to be massive. These large resource costs can be barriers in deploying these models in clinics, globally. To address this, there are cogent efforts within the machine learning community to introduce notions of resource efficiency. For instance, using quantisation to alleviate memory consumption. While most of these methods are shown to reduce the resource utilisation, they could come at a cost in performance. In this work, we probe into the trade-off between resource consumption and performance, specifically, when dealing with models that are used in critical settings such as in clinics.
Average Probability of Error for Single Uniprior Index Coding over Rayleigh Fading Channel
Venue: arXiv cs.IT 2024
Year: 2024

Ong and Ho developed optimal linear index codes for single uniprior index coding problems (ICPs) by finding a spanning tree for each of the strongly connected components of the corresponding information-flow graphs, following which Thomas et al. considered the same class of ICPs over Rayleigh fading channel. They developed the min-max probability of error criterion for choosing an index code which minimized the probability of error at the receivers and showed that there always exist optimal linear index codes for which any receiver takes at most two transmissions to decode a requested message. Motivated by the above works, this paper considers single uniprior ICPs over Rayleigh fading channels for which minimizing average probability of error is shown to be a criterion for further selection of index codes. The optimal index code w.r.t this criterion is shown to be one that minimizes the total number of transmissions used for decoding the message requests at all the receivers. An algorithm that generates a spanning tree which has a lower value of this metric as compared to the optimal star graph is also presented. For a given set of parameters of single uniprior ICPs, a lower bound for the total number of transmissions used by any optimal index code is derived, and a class of ICPs for which this bound is tight is identified.
RN-Net: Reservoir Nodes-Enabled Neuromorphic Vision Sensing Network
Venue: arXiv cs.CV 2024
Year: 2024

Event-based cameras are inspired by the sparse and asynchronous spike representation of the biological visual system. However, processing the event data requires either using expensive feature descriptors to transform spikes into frames, or using spiking neural networks that are expensive to train. In this work, we propose a neural network architecture, Reservoir Nodes-enabled neuromorphic vision sensing Network (RN-Net), based on simple convolution layers integrated with dynamic temporal encoding reservoirs for local and global spatiotemporal feature detection with low hardware and training costs. The RN-Net allows efficient processing of asynchronous temporal features, and achieves the highest accuracy of 99.2% for DVS128 Gesture reported to date, and one of the highest accuracy of 67.5% for DVS Lip dataset at a much smaller network size. By leveraging the internal device and circuit dynamics, asynchronous temporal feature encoding can be implemented at very low hardware cost without preprocessing and dedicated memory and arithmetic units. The use of simple DNN blocks and standard backpropagation-based training rules further reduces implementation costs.
Monocular Cyclist Detection with Convolutional Neural Networks
Venue: arXiv cs.CV 2024
Year: 2024

Cycling is an increasingly popular method of transportation for sustainability and health benefits. However, cyclists face growing risks, especially when encountering large vehicles on the road. This study aims to reduce the number of vehicle-cyclist collisions, which are often caused by poor driver attention to blind spots. To achieve this, we designed a state-of-the-art real-time monocular cyclist detection that can detect cyclists with object detection convolutional neural networks, such as EfficientDet Lite and SSD MobileNetV2. First, our proposed cyclist detection models achieve greater than 0.900 mAP (IoU: 0.5), fine-tuned on a newly proposed cyclist image dataset comprising over 20,000 images. Next, the models were deployed onto a Google Coral Dev Board mini-computer with a camera module and analyzed for speed, reaching inference times as low as 15 milliseconds. Lastly, the end-to-end cyclist detection device was tested in real-time to model traffic scenarios and analyzed further for performance and feasibility. We concluded that this cyclist detection device can accurately and quickly detect cyclists and has the potential to improve cyclist safety significantly. Future studies could determine the feasibility of the proposed device in the vehicle industry and improvements to cyclist safety over time.
Creating Ensembles of Classifiers through UMDA for Aerial Scene Classification
Venue: arXiv cs.CV 2024
Year: 2024

Aerial scene classification, which aims to semantically label remote sensing images in a set of predefined classes (e.g., agricultural, beach, and harbor), is a very challenging task in remote sensing due to high intra-class variability and the different scales and orientations of the objects present in the dataset images. In remote sensing area, the use of CNN architectures as an alternative solution is also a reality for scene classification tasks. Generally, these CNNs are used to perform the traditional image classification task. However, another less used way to classify remote sensing image might be the one that uses deep metric learning (DML) approaches. In this sense, this work proposes to employ six DML approaches for aerial scene classification tasks, analysing their behave with four different pre-trained CNNs as well as combining them through the use of evolutionary computation algorithm (UMDA). In performed experiments, it is possible to observe than DML approaches can achieve the best classification results when compared to traditional pre-trained CNNs for three well-known remote sensing aerial scene datasets. In addition, the UMDA algorithm proved to be a promising strategy to combine DML approaches when there is diversity among them, managing to improve at least 5.6% of accuracy in the classification results using almost 50\% of the available classifiers for the construction of the final ensemble of classifiers.
Studying How to Efficiently and Effectively Guide Models with Explanations
Venue: arXiv cs.CV 2024
Year: 2024

Despite being highly performant, deep neural networks might base their decisions on features that spuriously correlate with the provided labels, thus hurting generalization. To mitigate this, 'model guidance' has recently gained popularity, i.e. the idea of regularizing the models' explanations to ensure that they are "right for the right reasons". While various techniques to achieve such model guidance have been proposed, experimental validation of these approaches has thus far been limited to relatively simple and / or synthetic datasets. To better understand the effectiveness of the various design choices that have been explored in the context of model guidance, in this work we conduct an in-depth evaluation across various loss functions, attribution methods, models, and 'guidance depths' on the PASCAL VOC 2007 and MS COCO 2014 datasets. As annotation costs for model guidance can limit its applicability, we also place a particular focus on efficiency. Specifically, we guide the models via bounding box annotations, which are much cheaper to obtain than the commonly used segmentation masks, and evaluate the robustness of model guidance under limited (e.g. with only 1% of annotated images) or overly coarse annotations. Further, we propose using the EPG score as an additional evaluation metric and loss function ('Energy loss'). We show that optimizing for the Energy loss leads to models that exhibit a distinct focus on object-specific features, despite only using bounding box annotations that also include background regions. Lastly, we show that such model guidance can improve generalization under distribution shifts. Code available at: https://github.com/sukrutrao/Model-Guidance.
Logical Reasoning over Natural Language as Knowledge Representation: A Survey
Venue: arXiv cs.CL 2024
Year: 2024

Logical reasoning is central to human cognition and intelligence. It includes deductive, inductive, and abductive reasoning. Past research of logical reasoning within AI uses formal language as knowledge representation and symbolic reasoners. However, reasoning with formal language has proved challenging (e.g., brittleness and knowledge-acquisition bottleneck). This paper provides a comprehensive overview on a new paradigm of logical reasoning, which uses natural language as knowledge representation and pretrained language models as reasoners, including philosophical definition and categorization of logical reasoning, advantages of the new paradigm, benchmarks and methods, challenges of the new paradigm, possible future directions, and relation to related NLP fields. This new paradigm is promising since it not only alleviates many challenges of formal representation but also has advantages over end-to-end neural methods. This survey focus on transformer-based LLMs explicitly working on deductive, inductive, and abductive reasoning over English representation.
Roots and Requirements for Collaborative AIs
Venue: arXiv cs.AI 2024
Year: 2024

The vision of AI collaborators is a staple of mythology and science fiction, where artificial agents with special talents assist human partners and teams. In this dream, sophisticated AIs understand nuances of collaboration and human communication. The AI as collaborator dream is different from computer tools that augment human intelligence (IA) or intermediate human collaboration. Those tools have their roots in the 1960s and helped to drive an information technology revolution. They can be useful but they are not intelligent and do not collaborate as effectively as skilled people. With the increase of hybrid and remote work since the COVID pandemic, the benefits and requirements for better coordination, collaboration, and communication are becoming hot topics in the workplace. Employers and workers face choices and trade-offs as they negotiate the options for working from home versus working at the office. Many factors such as the high costs of homes near employers are impeding a mass return to the office. Government advisory groups and leaders in AI have advocated for years that AIs should be transparent and effective collaborators. Nonetheless, robust AIs that collaborate like talented people remain out of reach. Are AI teammates part of a solution? How artificially intelligent (AI) could and should they be? This position paper reviews the arc of technology and public calls for human-machine teaming. It draws on earlier research in psychology and the social sciences about what human-like collaboration requires. This paper sets a context for a second science-driven paper that advocates a radical shift in technology and methodology for creating resilient, intelligent, and human-compatible AIs (Stefik & Price, 2023). The aspirational goal is that such AIs would learn, share what they learn, and collaborate to achieve high capabilities.