OpenRaiser/Intern-Atlas huggingface dataset:
Emergent properties with repeated examples
Venue: arXiv cs.LG 2024
Year: 2024

We study the performance of transformers as a function of the number of repetitions of training examples with algorithmically generated datasets. On three problems of mathematics: the greatest common divisor, modular multiplication, and matrix eigenvalues, we show that for a fixed number of training steps, models trained on smaller sets of repeated examples outperform models trained on larger sets of single-use examples. We also demonstrate that two-set training - repeated use of a small random subset of examples, along normal sampling on the rest of the training set - provides for faster learning and better performance. This highlights that the benefits of repetition can outweigh those of data diversity. These datasets and problems provide a controlled setting to shed light on the still poorly understood interplay between generalization and memorization in deep learning.
Robots in the Middle: Evaluating LLMs in Dispute Resolution
Venue: arXiv cs.HC 2024
Year: 2024

Mediation is a dispute resolution method featuring a neutral third-party (mediator) who intervenes to help the individuals resolve their dispute. In this paper, we investigate to which extent large language models (LLMs) are able to act as mediators. We investigate whether LLMs are able to analyze dispute conversations, select suitable intervention types, and generate appropriate intervention messages. Using a novel, manually created dataset of 50 dispute scenarios, we conduct a blind evaluation comparing LLMs with human annotators across several key metrics. Overall, the LLMs showed strong performance, even outperforming our human annotators across dimensions. Specifically, in 62% of the cases, the LLMs chose intervention types that were rated as better than or equivalent to those chosen by humans. Moreover, in 84% of the cases, the intervention messages generated by the LLMs were rated as better than or equal to the intervention messages written by humans. LLMs likewise performed favourably on metrics such as impartiality, understanding and contextualization. Our results demonstrate the potential of integrating AI in online dispute resolution (ODR) platforms.
Automated and Complete Generation of Traffic Scenarios at Road Junctions Using a Multi-level Danger Definition
Venue: arXiv cs.SE 2024
Year: 2024

To ensure their safe use, autonomous vehicles (AVs) must meet rigorous certification criteria that involve executing maneuvers safely within (arbitrary) scenarios where other actors perform their intended maneuvers. For that purpose, existing scenario generation approaches optimize search to derive scenarios with high probability of dangerous situations. In this paper, we hypothesize that at road junctions, potential danger predominantly arises from overlapping paths of individual actors carrying out their designated high-level maneuvers. As a step towards AV certification, we propose an approach to derive a complete set of (potentially dangerous) abstract scenarios at any given road junction, i.e. all permutations of overlapping abstract paths assigned to actors (including the AV) for a given set of possible abstract paths. From these abstract scenarios, we derive exact paths that actors must follow to guide simulation-based testing towards potential collisions. We conduct extensive experiments to evaluate the behavior of a state-of-the-art learning-based AV controller on scenarios generated over two realistic road junctions with increasing number of external actors. Results show that the AV-under-test is involved in increasing percentages of unsafe behaviors in simulation, which vary according to functional- and logical-level scenario properties.
Classification of Buried Objects from Ground Penetrating Radar Images by using Second Order Deep Learning Models
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, a new classification model based on covariance matrices is built in order to classify buried objects. The inputs of the proposed models are the hyperbola thumbnails obtained with a classical Ground Penetrating Radar (GPR) system. These thumbnails are then inputs to the first layers of a classical CNN, which then produces a covariance matrix using the outputs of the convolutional filters. Next, the covariance matrix is given to a network composed of specific layers to classify Symmetric Positive Definite (SPD) matrices. We show in a large database that our approach outperform shallow networks designed for GPR data and conventional CNNs typically used in computer vision applications, particularly when the number of training data decreases and in the presence of mislabeled data. We also illustrate the interest of our models when training data and test sets are obtained from different weather modes or considerations.
A Realistic Model Reference Computed Torque Control Strategy for Human Lower Limb Exoskeletons
Venue: arXiv cs.RO 2024
Year: 2024

Exoskeleton robots have become a promising tool in neurorehabilitation, offering effective physical therapy and recovery monitoring. The success of these therapies relies on precise motion control systems. Although computed torque control based on inverse dynamics provides a robust theoretical foundation, its practical application in rehabilitation is limited by its sensitivity to model accuracy, making it less effective when dealing with unpredictable payloads. To overcome these limitations, this study introduces a novel model reference computed torque controller that accounts for parametric uncertainties while optimizing computational efficiency. A dynamic model of a seven-degree-of-freedom human lower limb exoskeleton is developed, incorporating a realistic joint friction model to accurately reflect the physical behavior of the robot. To reduce computational demands, the control system is split into two loops: a slower loop that predicts joint torque requirements based on input trajectories and robot dynamics, and a faster PID loop that corrects trajectory tracking errors. Coriolis and centrifugal forces are excluded from the model due to their minimal impact on system dynamics relative to their computational cost. Experimental results show high accuracy in trajectory tracking, and statistical analyses confirm the controller's robustness and effectiveness in handling parametric uncertainties. This approach presents a promising advancement for improving the stability and performance of exoskeleton-based neurorehabilitation.
CKMImageNet: A Comprehensive Dataset to Enable Channel Knowledge Map Construction via Computer Vision
Venue: arXiv cs.IT 2024
Year: 2024

Environment-aware communication and sensing is one of the promising paradigm shifts towards 6G, which fully leverages prior information of the local wireless environment to optimize network performance. One of the key enablers for environment-aware communication and sensing is channel knowledge map (CKM), which provides location-specific channel knowledge that is crucial for channel state information (CSI) acquisition. To support the efficient construction of CKM, large-scale location-specific channel data is essential. However, most existing channel datasets do not have the location information nor visual representations of channel data, making them inadequate for exploring the intrinsic relationship between the channel knowledge and the local environment, nor for applying advanced artificial intelligence (AI) algorithms such as computer vision (CV) for CKM construction. To address such issues, in this paper, a large-scale dataset named CKMImageNet is established, which can provide both location-tagged numerical channel data and visual images, providing a holistic view of the channel and environment. Built using commercial ray tracing software, CKMImageNet captures electromagnetic wave propagation in different scenarios, revealing the relationships between location, environment and channel knowledge. By integrating detailed channel data and the corresponding image, CKMImageNet not only supports the verification of various communication and sensing algorithms, but also enables CKM construction with CV algorithms.
vailá: Versatile Anarcho Integrated Liberation Ánalysis in Multimodal Toolbox
Venue: arXiv cs.HC 2024
Year: 2024

Human movement analysis is crucial in health and sports biomechanics for understanding physical performance, guiding rehabilitation, and preventing injuries. However, existing tools are often proprietary, expensive, and function as "black boxes", limiting user control and customization. This paper introduces vailá-Versatile Anarcho Integrated Liberation Ánalysis in Multimodal Toolbox-an open-source, Python-based platform designed to enhance human movement analysis by integrating data from multiple biomechanical systems. vailá supports data from diverse sources, including retroreflective motion capture systems, inertial measurement units (IMUs), markerless video capture technology, electromyography (EMG), force plates, and GPS or GNSS systems, enabling comprehensive analysis of movement patterns. Developed entirely in Python 3.11.9, which offers improved efficiency and long-term support, and featuring a straightforward installation process, vailá is accessible to users without extensive programming experience. In this paper, we also present several workflow examples that demonstrate how vailá allows the rapid processing of large batches of data, independent of the type of collection method. This flexibility is especially valuable in research scenarios where unexpected data collection challenges arise, ensuring no valuable data point is lost. We demonstrate the application of vailá in analyzing sit-to-stand movements in pediatric disability, showcasing its capability to provide deeper insights even with unexpected movement patterns. By fostering a collaborative and open environment, vailá encourages users to innovate, customize, and freely explore their analysis needs, potentially contributing to the advancement of rehabilitation strategies and performance optimization.
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
Venue: arXiv cs.AR 2024
Year: 2024

The rapid development of large language models (LLMs) has significantly transformed the field of artificial intelligence, demonstrating remarkable capabilities in natural language processing and moving towards multi-modal functionality. These models are increasingly integrated into diverse applications, impacting both research and industry. However, their development and deployment present substantial challenges, including the need for extensive computational resources, high energy consumption, and complex software optimizations. Unlike traditional deep learning systems, LLMs require unique optimization strategies for training and inference, focusing on system-level efficiency. This paper surveys hardware and software co-design approaches specifically tailored to address the unique characteristics and constraints of large language models. This survey analyzes the challenges and impacts of LLMs on hardware and algorithm research, exploring algorithm optimization, hardware design, and system-level innovations. It aims to provide a comprehensive understanding of the trade-offs and considerations in LLM-centric computing systems, guiding future advancements in AI. Finally, we summarize the existing efforts in this space and outline future directions toward realizing production-grade co-design methodologies for the next generation of large language models and AI systems.
Learning Content-Aware Multi-Modal Joint Input Pruning via Bird's-Eye-View Representation
Venue: arXiv cs.CV 2024
Year: 2024

In the landscape of autonomous driving, Bird's-Eye-View (BEV) representation has recently garnered substantial academic attention, serving as a transformative framework for the fusion of multi-modal sensor inputs. This BEV paradigm effectively shifts the sensor fusion challenge from a rule-based methodology to a data-centric approach, thereby facilitating more nuanced feature extraction from an array of heterogeneous sensors. Notwithstanding its evident merits, the computational overhead associated with BEV-based techniques often mandates high-capacity hardware infrastructures, thus posing challenges for practical, real-world implementations. To mitigate this limitation, we introduce a novel content-aware multi-modal joint input pruning technique. Our method leverages BEV as a shared anchor to algorithmically identify and eliminate non-essential sensor regions prior to their introduction into the perception model's backbone. We validatethe efficacy of our approach through extensive experiments on the NuScenes dataset, demonstrating substantial computational efficiency without sacrificing perception accuracy. To the best of our knowledge, this work represents the first attempt to alleviate the computational burden from the input pruning point.
ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Model
Venue: arXiv cs.CV 2024
Year: 2024

Generating human motion from textual descriptions is a challenging task. Existing methods either struggle with physical credibility or are limited by the complexities of physics simulations. In this paper, we present \emph{ReinDiffuse} that combines reinforcement learning with motion diffusion model to generate physically credible human motions that align with textual descriptions. Our method adapts Motion Diffusion Model to output a parameterized distribution of actions, making them compatible with reinforcement learning paradigms. We employ reinforcement learning with the objective of maximizing physically plausible rewards to optimize motion generation for physical fidelity. Our approach outperforms existing state-of-the-art models on two major datasets, HumanML3D and KIT-ML, achieving significant improvements in physical plausibility and motion quality. Project: https://reindiffuse.github.io/
Autonomous Navigation and Collision Avoidance for Mobile Robots: Classification and Review
Venue: arXiv cs.RO 2024
Year: 2024

This paper introduces a novel classification for Autonomous Mobile Robots (AMRs), into three phases and five steps, focusing on autonomous collision-free navigation. Additionally, it presents the main methods and widely accepted technologies for each phase of the proposed classification. The purpose of this classification is to facilitate understanding and establish connections between the independent input variables of the system (hardware, software) and autonomous navigation. By analyzing well-established technologies in terms of sensors and methods used for autonomous navigation, this paper aims to provide a foundation of knowledge that can be applied in future projects of mobile robots.
Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow
Venue: arXiv cs.CV 2024
Year: 2024

Diffusion models have greatly improved visual generation but are hindered by slow generation speed due to the computationally intensive nature of solving generative ODEs. Rectified flow, a widely recognized solution, improves generation speed by straightening the ODE path. Its key components include: 1) using the diffusion form of flow-matching, 2) employing $\boldsymbol v$-prediction, and 3) performing rectification (a.k.a. reflow). In this paper, we argue that the success of rectification primarily lies in using a pretrained diffusion model to obtain matched pairs of noise and samples, followed by retraining with these matched noise-sample pairs. Based on this, components 1) and 2) are unnecessary. Furthermore, we highlight that straightness is not an essential training target for rectification; rather, it is a specific case of flow-matching models. The more critical training target is to achieve a first-order approximate ODE path, which is inherently curved for models like DDPM and Sub-VP. Building on this insight, we propose Rectified Diffusion, which generalizes the design space and application scope of rectification to encompass the broader category of diffusion models, rather than being restricted to flow-matching models. We validate our method on Stable Diffusion v1-5 and Stable Diffusion XL. Our method not only greatly simplifies the training procedure of rectified flow-based previous works (e.g., InstaFlow) but also achieves superior performance with even lower training cost. Our code is available at https://github.com/G-U-N/Rectified-Diffusion.
Optimal-Length Labeling Schemes for Fast Deterministic Communication in Radio Networks
Venue: arXiv cs.DC 2024
Year: 2024

We consider two fundamental communication tasks in arbitrary radio networks: broadcasting (information from one source has to reach all nodes) and gossiping (every node has a message and all messages have to reach all nodes). Nodes are assigned labels that are (not necessarily different) binary strings. Each node knows its own label and can use it as a parameter in the same deterministic algorithm. The length of a labeling scheme is the largest length of a label. The goal is to find labeling schemes of asymptotically optimal length for the above tasks, and to design fast deterministic distributed algorithms for each of them, using labels of optimal length. Our main result concerns broadcasting. We show the existence of a labeling scheme of constant length that supports broadcasting in time $O(D+\log^2 n)$, where $D$ is the diameter of the network and $n$ is the number of nodes. This broadcasting time is an improvement over the best currently known $O(D\log n + \log^2 n)$ time of broadcasting with constant-length labels, due to Ellen and Gilbert (SPAA 2020). It also matches the optimal broadcasting time in radio networks of known topology. Hence, we show that appropriately chosen node labels of constant length permit to achieve, in a distributed way, the optimal centralized broadcasting time. This is, perhaps, the most surprising finding of this paper. We are able to obtain our result thanks to a novel methodological tool of propagating information in radio networks, that we call a 2-height respecting tree. Next, we apply our broadcasting algorithm to solve the gossiping problem. We get a gossiping algorithm working in time $O(D + Δ\log n + \log^2 n)$, using a labeling scheme of optimal length $O(\log Δ)$, where $Δ$ is the maximum degree. Our time is the same as the best known gossiping time in radio networks of known topology.
LLM Embeddings Improve Test-time Adaptation to Tabular $Y|X$-Shifts
Venue: arXiv cs.LG 2024
Year: 2024

For tabular datasets, the change in the relationship between the label and covariates ($Y|X$-shifts) is common due to missing variables (a.k.a. confounders). Since it is impossible to generalize to a completely new and unknown domain, we study models that are easy to adapt to the target domain even with few labeled examples. We focus on building more informative representations of tabular data that can mitigate $Y|X$-shifts, and propose to leverage the prior world knowledge in LLMs by serializing (write down) the tabular data to encode it. We find LLM embeddings alone provide inconsistent improvements in robustness, but models trained on them can be well adapted/finetuned to the target domain even using 32 labeled observations. Our finding is based on a comprehensive and systematic study consisting of 7650 source-target pairs and benchmark against 261,000 model configurations trained by 22 algorithms. Our observation holds when ablating the size of accessible target data and different adaptation strategies. The code is available at https://github.com/namkoong-lab/LLM-Tabular-Shifts.
Aligning AI-driven discovery with human intuition
Venue: arXiv cs.LG 2024
Year: 2024

As data-driven modeling of physical dynamical systems becomes more prevalent, a new challenge is emerging: making these models more compatible and aligned with existing human knowledge. AI-driven scientific modeling processes typically begin with identifying hidden state variables, then deriving governing equations, followed by predicting and analyzing future behaviors. The critical initial step of identification of an appropriate set of state variables remains challenging for two reasons. First, finding a compact set of meaningfully predictive variables is mathematically difficult and under-defined. A second reason is that variables found often lack physical significance, and are therefore difficult for human scientists to interpret. We propose a new general principle for distilling representations that are naturally more aligned with human intuition, without relying on prior physical knowledge. We demonstrate our approach on a number of experimental and simulated system where the variables generated by the AI closely resemble those chosen independently by human scientists. We suggest that this principle can help make human-AI collaboration more fruitful, as well as shed light on how humans make scientific modeling choices.
On the Feasibility of A Mixed-Method Approach for Solving Long Horizon Task-Oriented Dexterous Manipulation
Venue: arXiv cs.RO 2024
Year: 2024

In-hand manipulation of tools using dexterous hands in real-world is an underexplored problem in the literature. In addition to more complex geometry and larger size of the tools compared to more commonly used objects like cubes or cylinders, task oriented in-hand tool manipulation involves many sub-tasks to be performed sequentially. This may involve reaching to the tool, picking it up, reorienting it in hand with or without regrasping to reach to a desired final grasp appropriate for the tool usage, and carrying the tool to the desired pose. Research on long-horizon manipulation using dexterous hands is rather limited and the existing work focus on learning the individual sub-tasks using a method like reinforcement learning (RL) and combine the policies for different subtasks to perform a long horizon task. However, in general a single method may not be the best for all the sub-tasks, and this can be more pronounced when dealing with multi-fingered hands manipulating objects with complex geometry like tools. In this paper, we investigate the use of a mixed-method approach to solve for the long-horizon task of tool usage and we use imitation learning, reinforcement learning and model based control. We also discuss a new RL-based teacher-student framework that combines real world data into offline training. We show that our proposed approach for each subtask outperforms the commonly adopted reinforcement learning approach across different subtasks and in performing the long horizon task in simulation. Finally we show the successful transferability to real world.
Aligning Motion-Blurred Images Using Contrastive Learning on Overcomplete Pixels
Venue: arXiv cs.CV 2024
Year: 2024

We propose a new contrastive objective for learning overcomplete pixel-level features that are invariant to motion blur. Other invariances (e.g., pose, illumination, or weather) can be learned by applying the corresponding transformations on unlabeled images during self-supervised training. We showcase that a simple U-Net trained with our objective can produce local features useful for aligning the frames of an unseen video captured with a moving camera under realistic and challenging conditions. Using a carefully designed toy example, we also show that the overcomplete pixels can encode the identity of objects in an image and the pixel coordinates relative to these objects.