OpenRaiser/Intern-Atlas huggingface dataset:
Self-Sensing for Proprioception and Contact Detection in Soft Robots Using Shape Memory Alloy Artificial Muscles
Venue: arXiv cs.RO 2024
Year: 2024

Estimating a soft robot's pose and applied forces, also called proprioception, is crucial for safe interaction of the robot with its environment. However, most solutions for soft robot proprioception use dedicated sensors, particularly for external forces, which introduce design trade-offs, rigidity, and risk of failure. This work presents an approach for pose estimation and contact detection for soft robots actuated by shape memory alloy (SMA) artificial muscles, using no dedicated force sensors. Our framework uses the unique material properties of SMAs to self-sense their internal stress, via offboard measurements of their electrical resistance and in-situ temperature readings, in an existing fully-soft limb design. We demonstrate that a simple polynomial regression model on these measurements is sufficient to predict the robot's pose, under no-contact conditions. Then, we show that if an additional measurement of the true pose is available (e.g. from an already-in-place bending sensor), it is possible to predict a binary contact/no-contact using multiple combinations of self-sensing signals. Our hardware tests verify our hypothesis via a contact detection test with a human operator. This proof-of-concept validates that self-sensing signals in soft SMA-actuated soft robots can be used for proprioception and contact detection, and suggests a direction for integrating proprioception into soft robots without design compromises. Future work could employ machine learning for enhanced accuracy.
NetScaNDN: A Scalable and Flexible Testbed To Evaluate NDN on Multiple Infrastructures
Venue: arXiv cs.NI 2024
Year: 2024

The evolution from traditional IP-based networking to Named Data Networking (NDN) represents a paradigm shift to address the inherent limitations of current network architectures, such as scalability, mobility, and efficient data distribution. NDN introduces an information-centric approach where data is identified and retrieved based on names rather than locations, offering more efficient data dissemination and enhanced security. However, the transition to NDN, alongside the need to integrate it with existing IP infrastructures, necessitates the development of flexible and scalable testbeds that support diverse experimental scenarios across various physical media and networking protocol stacks. In this paper, we present NetScaNDN, a scalable, flexible, and plug-and-play testbed designed to facilitate such experiments. NetScaNDNl employs an automated process for node discovery, configuration, and installation, enabling seamless setup and execution of experiments on both wired and wireless infrastructures simultaneously. Additionally, it incorporates a central log repository using the syslog protocol, allowing comprehensive measurement and evaluation of user-defined metrics across different network layers. NetScaNDN offers a robust platform for researchers to explore and validate various networking scenarios, advancing the study of IP and NDN-based applications.
Differential Privacy Regularization: Protecting Training Data Through Loss Function Regularization
Venue: arXiv cs.LG 2024
Year: 2024

Training machine learning models based on neural networks requires large datasets, which may contain sensitive information. The models, however, should not expose private information from these datasets. Differentially private SGD [DP-SGD] requires the modification of the standard stochastic gradient descent [SGD] algorithm for training new models. In this short paper, a novel regularization strategy is proposed to achieve the same goal in a more efficient manner.
BERTScoreVisualizer: A Web Tool for Understanding Simplified Text Evaluation with BERTScore
Venue: arXiv cs.CL 2024
Year: 2024

The BERTScore metric is commonly used to evaluate automatic text simplification systems. However, current implementations of the metric fail to provide complete visibility into all information the metric can produce. Notably, the specific token matchings can be incredibly useful in generating clause-level insight into the quality of simplified text. We address this by introducing BERTScoreVisualizer, a web application that goes beyond reporting precision, recall, and F1 score and provides a visualization of the matching between tokens. We believe that our software can help improve the analysis of text simplification systems by specifically showing where generated, simplified text deviates from reference text. We host our code and demo on GitHub.
Autonomous Vehicle Decision-Making Framework for Considering Malicious Behavior at Unsignalized Intersections
Venue: arXiv cs.RO 2024
Year: 2024

In this paper, we propose a Q-learning based decision-making framework to improve the safety and efficiency of Autonomous Vehicles when they encounter other maliciously behaving vehicles while passing through unsignalized intersections. In Autonomous Vehicles, conventional reward signals are set as regular rewards regarding feedback factors such as safety and efficiency. In this paper, safety gains are modulated by variable weighting parameters to ensure that safety can be emphasized more in emergency situations. The framework proposed in this paper introduces first-order theory of mind inferences on top of conventional rewards, using first-order beliefs as additional reward signals. The decision framework enables Autonomous Vehicles to make informed decisions when encountering vehicles with potentially malicious behaviors at unsignalized intersections, thereby improving the overall safety and efficiency of Autonomous Vehicle transportation systems. In order to verify the performance of the decision framework, this paper uses Prescan/Simulink co-simulations for simulation, and the results show that the performance of the decision framework can meet the set requirements.
Towards Using Active Learning Methods for Human-Seat Interactions To Generate Realistic Occupant Motion
Venue: arXiv cs.RO 2024
Year: 2024

In the context of developing new vehicle concepts, especially autonomous vehicles with novel seating arrangements and occupant activities, predicting occupant motion can be a tool for ensuring safety and comfort. In this study, a data-driven surrogate contact model integrated into an optimal control framework to predict human occupant behavior during driving maneuvers is presented. High-fidelity finite element simulations are utilized to generate a dataset of interaction forces and moments for various human body configurations and velocities. To automate the generation of training data, an active learning approach is introduced, which iteratively queries the high-fidelity finite element simulation for an additional dataset. The feasibility and effectiveness of the proposed method are demonstrated through a case study of a head interaction with an automotive headrest, showing promising results in accurately replicating contact forces and moments while reducing manual effort.
ScriptSmith: A Unified LLM Framework for Enhancing IT Operations via Automated Bash Script Generation, Assessment, and Refinement
Venue: arXiv cs.SE 2024
Year: 2024

In the rapidly evolving landscape of site reliability engineering (SRE), the demand for efficient and effective solutions to manage and resolve issues in site and cloud applications is paramount. This paper presents an innovative approach to action automation using large language models (LLMs) for script generation, assessment, and refinement. By leveraging the capabilities of LLMs, we aim to significantly reduce the human effort involved in writing and debugging scripts, thereby enhancing the productivity of SRE teams. Our experiments focus on Bash scripts, a commonly used tool in SRE, and involve the CodeSift dataset of 100 tasks and the InterCode dataset of 153 tasks. The results show that LLMs can automatically assess and refine scripts efficiently, reducing the need for script validation in an execution environment. Results demonstrate that the framework shows an overall improvement of 7-10% in script generation.
Enhancing Guardrails for Safe and Secure Healthcare AI
Venue: arXiv cs.CR 2024
Year: 2024

Generative AI holds immense promise in addressing global healthcare access challenges, with numerous innovative applications now ready for use across various healthcare domains. However, a significant barrier to the widespread adoption of these domain-specific AI solutions is the lack of robust safety mechanisms to effectively manage issues such as hallucination, misinformation, and ensuring truthfulness. Left unchecked, these risks can compromise patient safety and erode trust in healthcare AI systems. While general-purpose frameworks like Llama Guard are useful for filtering toxicity and harmful content, they do not fully address the stringent requirements for truthfulness and safety in healthcare contexts. This paper examines the unique safety and security challenges inherent to healthcare AI, particularly the risk of hallucinations, the spread of misinformation, and the need for factual accuracy in clinical settings. I propose enhancements to existing guardrails frameworks, such as Nvidia NeMo Guardrails, to better suit healthcare-specific needs. By strengthening these safeguards, I aim to ensure the secure, reliable, and accurate use of AI in healthcare, mitigating misinformation risks and improving patient safety.
A random measure approach to reinforcement learning in continuous time
Venue: arXiv cs.LG 2024
Year: 2024

We present a random measure approach for modeling exploration, i.e., the execution of measure-valued controls, in continuous-time reinforcement learning (RL) with controlled diffusion and jumps. First, we consider the case when sampling the randomized control in continuous time takes place on a discrete-time grid and reformulate the resulting stochastic differential equation (SDE) as an equation driven by suitable random measures. The construction of these random measures makes use of the Brownian motion and the Poisson random measure (which are the sources of noise in the original model dynamics) as well as the additional random variables, which are sampled on the grid for the control execution. Then, we prove a limit theorem for these random measures as the mesh-size of the sampling grid goes to zero, which leads to the grid-sampling limit SDE that is jointly driven by white noise random measures and a Poisson random measure. We also argue that the grid-sampling limit SDE can substitute the exploratory SDE and the sample SDE of the recent continuous-time RL literature, i.e., it can be applied for the theoretical analysis of exploratory control problems and for the derivation of learning algorithms.
Exploring the Roles of NLP-based Dialog Indicators in Predicting User Experience in interacting with Large Language Model System
Venue: arXiv cs.HC 2024
Year: 2024

The use of Large Language Models for dialogue systems is rising, presenting a new challenge: how do we assess users' chat experience in these systems? Leveraging Natural Language Processing (NLP)-powered dialog analyzers to create dialog indicators like Coherence and Emotion has the potential to predict the chat experience. In this paper, we proposed a conceptual model to explain the relationship between the dialog indicators and various factors related to the chat experience, such as users' intentions, affinity toward dialog agents, and prompts of the agents' characters. We evaluated the conceptual model using PLS-SEM with 120 participants and found it well fit. Our results suggest that dialog indicators can predict the chat experience and fully mediate the impact of prompts and user intentions. Additionally, users' affinity toward agents can partially explain these predictions. Our findings demonstrate the potential of using dialog indicators in predicting the chat experience. Through the conceptual model we propose, researchers can apply the dialog analyzers to generate dialog indicators to constantly monitor the dialog process and improve the user's chat experience accordingly.
Finite State Machine with Input and Process Render
Venue: arXiv cs.CY 2024
Year: 2024

Finite State Machines are a concept widely taught in undergraduate theory of computing courses. Educators typically use tools with static representations of FSMs to help students visualize these objects and processes; however, all existing tools require manual editing by the instructor. In this poster, we created an automatic visualization tool for FSMs that generates videos of FSM simulation, named Finite State Machine with Input and Process Render (FSMIPR). Educators can input any formal definition of an FSM and an input string, and FSMIPR generates an accompanying video of its simulation. We believe that FSMIPR will be beneficial to students who learn difficult computer theory concepts. We conclude with future work currently in-progress with FSMIPR.
Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoning
Venue: arXiv cs.AI 2024
Year: 2024

Large Language Models (LLMs) have revolutionized natural language processing, yet they struggle with inconsistent reasoning, particularly in novel domains and complex logical sequences. This research introduces Proof of Thought, a framework that enhances the reliability and transparency of LLM outputs. Our approach bridges LLM-generated ideas with formal logic verification, employing a custom interpreter to convert LLM outputs into First Order Logic constructs for theorem prover scrutiny. Central to our method is an intermediary JSON-based Domain-Specific Language, which by design balances precise logical structures with intuitive human concepts. This hybrid representation enables both rigorous validation and accessible human comprehension of LLM reasoning processes. Key contributions include a robust type system with sort management for enhanced logical integrity, explicit representation of rules for clear distinction between factual and inferential knowledge, and a flexible architecture that allows for easy extension to various domain-specific applications. We demonstrate Proof of Thought's effectiveness through benchmarking on StrategyQA and a novel multimodal reasoning task, showing improved performance in open-ended scenarios. By providing verifiable and interpretable results, our technique addresses critical needs for AI system accountability and sets a foundation for human-in-the-loop oversight in high-stakes domains.
Steering LLM Summarization with Visual Workspaces for Sensemaking
Venue: arXiv cs.HC 2024
Year: 2024

Large Language Models (LLMs) have been widely applied in summarization due to their speedy and high-quality text generation. Summarization for sensemaking involves information compression and insight extraction. Human guidance in sensemaking tasks can prioritize and cluster relevant information for LLMs. However, users must translate their cognitive thinking into natural language to communicate with LLMs. Can we use more readable and operable visual representations to guide the summarization process for sensemaking? Therefore, we propose introducing an intermediate step--a schematic visual workspace for human sensemaking--before the LLM generation to steer and refine the summarization process. We conduct a series of proof-of-concept experiments to investigate the potential for enhancing the summarization by GPT-4 through visual workspaces. Leveraging a textual sensemaking dataset with a ground truth summary, we evaluate the impact of a human-generated visual workspace on LLM-generated summarization of the dataset and assess the effectiveness of space-steered summarization. We categorize several types of extractable information from typical human workspaces that can be injected into engineered prompts to steer the LLM summarization. The results demonstrate how such workspaces can help align an LLM with the ground truth, leading to more accurate summarization results than without the workspaces.
Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors
Venue: arXiv cs.CV 2024
Year: 2024

Current video-based computer vision (CV) applications typically suffer from high energy consumption due to reading and processing all pixels in a frame, regardless of their significance. While previous works have attempted to reduce this energy by skipping input patches or pixels and using feedback from the end task to guide the skipping algorithm, the skipping is not performed during the sensor read phase. As a result, these methods can not optimize the front-end sensor energy. Moreover, they may not be suitable for real-time applications due to the long latency of modern CV networks that are deployed in the back-end. To address this challenge, this paper presents a custom-designed reconfigurable CMOS image sensor (CIS) system that improves energy efficiency by selectively skipping uneventful regions or rows within a frame during the sensor's readout phase, and the subsequent analog-to-digital conversion (ADC) phase. A novel masking algorithm intelligently directs the skipping process in real-time, optimizing both the front-end sensor and back-end neural networks for applications including autonomous driving and augmented/virtual reality (AR/VR). Our system can also operate in standard mode without skipping, depending on application needs. We evaluate our hardware-algorithm co-design framework on object detection based on BDD100K and ImageNetVID, and gaze estimation based on OpenEDS, achieving up to 53% reduction in front-end sensor energy while maintaining state-of-the-art (SOTA) accuracy.
Revisiting inverse Hessian vector products for calculating influence functions
Venue: arXiv cs.LG 2024
Year: 2024

Influence functions are a popular tool for attributing a model's output to training data. The traditional approach relies on the calculation of inverse Hessian-vector products (iHVP), but the classical solver "Linear time Stochastic Second-order Algorithm" (LiSSA, Agarwal et al. (2017)) is often deemed impractical for large models due to expensive computation and hyperparameter tuning. We show that the three hyperparameters -- the scaling factor, the batch size, and the number of steps -- can be chosen depending on the spectral properties of the Hessian, particularly its trace and largest eigenvalue. By evaluating with random sketching (Swartworth and Woodruff, 2023), we find that the batch size has to be sufficiently large for LiSSA to converge; however, for all of the models we consider, the requirement is mild. We confirm our findings empirically by comparing to Proximal Bregman Retraining Functions (PBRF, Bae et al. (2022)). Finally, we discuss what role the inverse Hessian plays in calculating the influence.
Cat-and-Mouse Satellite Dynamics: Divergent Adversarial Reinforcement Learning for Contested Multi-Agent Space Operations
Venue: arXiv cs.RO 2024
Year: 2024

As space becomes increasingly crowded and contested, robust autonomous capabilities for multi-agent environments are gaining critical importance. Current autonomous systems in space primarily rely on optimization-based path planning or long-range orbital maneuvers, which have not yet proven effective in adversarial scenarios where one satellite is actively pursuing another. We introduce Divergent Adversarial Reinforcement Learning (DARL), a two-stage Multi-Agent Reinforcement Learning (MARL) approach designed to train autonomous evasion strategies for satellites engaged with multiple adversarial spacecraft. Our method enhances exploration during training by promoting diverse adversarial strategies, leading to more robust and adaptable evader models. We validate DARL through a cat-and-mouse satellite scenario, modeled as a partially observable multi-agent capture the flag game where two adversarial `cat' spacecraft pursue a single `mouse' evader. DARL's performance is compared against several benchmarks, including an optimization-based satellite path planner, demonstrating its ability to produce highly robust models for adversarial multi-agent space environments.
Enhancing Financial Sentiment Analysis with Expert-Designed Hint
Venue: arXiv cs.CL 2024
Year: 2024

This paper investigates the role of expert-designed hint in enhancing sentiment analysis on financial social media posts. We explore the capability of large language models (LLMs) to empathize with writer perspectives and analyze sentiments. Our findings reveal that expert-designed hint, i.e., pointing out the importance of numbers, significantly improve performances across various LLMs, particularly in cases requiring perspective-taking skills. Further analysis on tweets containing different types of numerical data demonstrates that the inclusion of expert-designed hint leads to notable improvements in sentiment analysis performance, especially for tweets with monetary-related numbers. Our findings contribute to the ongoing discussion on the applicability of Theory of Mind in NLP and open new avenues for improving sentiment analysis in financial domains through the strategic use of expert knowledge.
Tactile Probabilistic Contact Dynamics Estimation of Unknown Objects
Venue: arXiv cs.RO 2024
Year: 2024

We study the problem of rapidly identifying contact dynamics of unknown objects in partially known environments. The key innovation of our method is a novel formulation of the contact dynamics estimation problem as the joint estimation of contact geometries and physical parameters. We leverage DeepSDF, a compact and expressive neural-network-based geometry representation over a distribution of geometries, and adopt a particle filter to estimate both the geometries in contact and the physical parameters. In addition, we couple the estimator with an active exploration strategy that plans information-gathering moves to further expedite online estimation. Through simulation and physical experiments, we show that our method estimates accurate contact dynamics with fewer than 30 exploration moves for unknown objects touching partially known environments.
Traverse the Non-Traversable: Estimating Traversability for Wheeled Mobility on Vertically Challenging Terrain
Venue: arXiv cs.RO 2024
Year: 2024

Most traversability estimation techniques divide off-road terrain into traversable (e.g., pavement, gravel, and grass) and non-traversable (e.g., boulders, vegetation, and ditches) regions and then inform subsequent planners to produce trajectories on the traversable part. However, recent research demonstrated that wheeled robots can traverse vertically challenging terrain (e.g., extremely rugged boulders comparable in size to the vehicles themselves), which unfortunately would be deemed as non-traversable by existing techniques. Motivated by such limitations, this work aims at identifying the traversable from the seemingly non-traversable, vertically challenging terrain based on past kinodynamic vehicle-terrain interactions in a data-driven manner. Our new Traverse the Non-Traversable(TNT) traversability estimator can efficiently guide a down-stream sampling-based planner containing a high-precision 6-DoF kinodynamic model, which becomes deployable onboard a small-scale vehicle. Additionally, the estimated traversability can also be used as a costmap to plan global and local paths without sampling. Our experiment results show that TNT can improve planning performance, efficiency, and stability by 50%, 26.7%, and 9.2% respectively on a physical robot platform.