OpenRaiser/Intern-Atlas huggingface dataset:
Towards Probabilistic Planning of Explanations for Robot Navigation
Venue: arXiv cs.AI 2024
Year: 2024

In robotics, ensuring that autonomous systems are comprehensible and accountable to users is essential for effective human-robot interaction. This paper introduces a novel approach that integrates user-centered design principles directly into the core of robot path planning processes. We propose a probabilistic framework for automated planning of explanations for robot navigation, where the preferences of different users regarding explanations are probabilistically modeled to tailor the stochasticity of the real-world human-robot interaction and the communication of decisions of the robot and its actions towards humans. This approach aims to enhance the transparency of robot path planning and adapt to diverse user explanation needs by anticipating the types of explanations that will satisfy individual users.
Ultrasound-Based AI for COVID-19 Detection: A Comprehensive Review of Public and Private Lung Ultrasound Datasets and Studies
Venue: arXiv cs.CV 2024
Year: 2024

The COVID-19 pandemic has affected millions of people globally, with respiratory organs being strongly affected in individuals with comorbidities. Medical imaging-based diagnosis and prognosis have become increasingly popular in clinical settings for detecting COVID-19 lung infections. Among various medical imaging modalities, ultrasound stands out as a low-cost, mobile, and radiation-safe imaging technology. In this comprehensive review, we focus on AI-driven studies utilizing lung ultrasound (LUS) for COVID-19 detection and analysis. We provide a detailed overview of both publicly available and private LUS datasets and categorize the AI studies according to the dataset they used. Additionally, we systematically analyzed and tabulated the studies across various dimensions, including data preprocessing methods, AI models, cross-validation techniques, and evaluation metrics. In total, we reviewed 60 articles, 41 of which utilized public datasets, while the remaining employed private data. Our findings suggest that ultrasound-based AI studies for COVID-19 detection have great potential for clinical use, especially for children and pregnant women. Our review also provides a useful summary for future researchers and clinicians who may be interested in the field.
Multi-language Video Subtitle Dataset for Image-based Text Recognition
Venue: arXiv cs.CV 2024
Year: 2024

The Multi-language Video Subtitle Dataset is a comprehensive collection designed to support research in text recognition across multiple languages. This dataset includes 4,224 subtitle images extracted from 24 videos sourced from online platforms. It features a wide variety of characters, including Thai consonants, vowels, tone marks, punctuation marks, numerals, Roman characters, and Arabic numerals. With 157 unique characters, the dataset provides a resource for addressing challenges in text recognition within complex backgrounds. It addresses the growing need for high-quality, multilingual text recognition data, particularly as videos with embedded subtitles become increasingly dominant on platforms like YouTube and Facebook. The variability in text length, font, and placement within these images adds complexity, offering a valuable resource for developing and evaluating deep learning models. The dataset facilitates accurate text transcription from video content while providing a foundation for improving computational efficiency in text recognition systems. As a result, it holds significant potential to drive advancements in research and innovation across various computer science disciplines, including artificial intelligence, deep learning, computer vision, and pattern recognition.
PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
Venue: arXiv cs.CL 2024
Year: 2024

The interest in developing small language models (SLM) for on-device deployment is fast growing. However, the existing SLM design hardly considers the device hardware characteristics. Instead, this work presents a simple yet effective principle for SLM design: architecture searching for (near-)optimal runtime efficiency before pre-training. Guided by this principle, we develop PhoneLM SLM family (currently with 0.5B and 1.5B versions), that acheive the state-of-the-art capability-efficiency tradeoff among those with similar parameter size. We fully open-source the code, weights, and training datasets of PhoneLM for reproducibility and transparency, including both base and instructed versions. We also release a finetuned version of PhoneLM capable of accurate Android Intent invocation, and an end-to-end Android demo. All materials are available at https://github.com/UbiquitousLearning/PhoneLM.
FineTuneBench: How well do commercial fine-tuning APIs infuse knowledge into LLMs?
Venue: arXiv cs.CL 2024
Year: 2024

There is great interest in fine-tuning frontier large language models (LLMs) to inject new information and update existing knowledge. While commercial LLM fine-tuning APIs from providers such as OpenAI and Google promise flexible adaptation for various applications, the efficacy of fine-tuning remains unclear. In this study, we introduce FineTuneBench, an evaluation framework and dataset for understanding how well commercial fine-tuning APIs can successfully learn new and updated knowledge. We analyze five frontier LLMs with commercially available fine-tuning APIs, including GPT-4o and Gemini 1.5 Pro, on their effectiveness in two settings: (1) ingesting novel information, such as recent news events and new people profiles, and (2) updating existing knowledge, such as updated medical guidelines and code frameworks. Our results reveal substantial shortcomings in all the models' abilities to effectively learn new information through fine-tuning, with an average generalization accuracy of 37% across all models. When updating existing knowledge, such as incorporating medical guideline updates, commercial fine-tuning APIs show even more limited capability (average generalization accuracy of 19%). Overall, fine-tuning GPT-4o mini is the most effective for infusing new knowledge and updating knowledge, followed by GPT-3.5 Turbo and GPT-4o. The fine-tuning APIs for Gemini 1.5 Flesh and Gemini 1.5 Pro are unable to learn new knowledge or update existing knowledge. These findings underscore a major shortcoming in using current commercial fine-tuning services to achieve reliable knowledge infusion in common scenarios. We open source the FineTuneBench dataset at https://github.com/kevinwu23/StanfordFineTuneBench.
Exploring Vibrotactile Intensity Perception with Multiple Waveform Parameters
Venue: arXiv cs.HC 2024
Year: 2024

It is known that by longing the duration of a vibrotactile stimuli or applying a damping or an increasing factor to the waveform the perceived intensity is affected in different ways. This paper presents a vibrotactile presentation system assembled with a software waveform generator that enables comparison in the perceived intensity for different waveforms made by multiple parameters. The adjustable parameters are frequency, amplitude, and wave type for the basic part of the stimuli and in addition, it is possible to apply an exponential decay or increasing factor to the waveform by specificizing the duration. By using the presented system, an easy comparison of the influence to the perception of intensity by different parameters of the waveform is possible. We conducted a preliminary experiment on a variety of waveshapes with and without damping by using this system.
EnchantedClothes: Visual and Tactile Feedback with an Abdomen-Attached Robot through Clothes
Venue: arXiv cs.RO 2024
Year: 2024

Wearable robots are designed to be worn on the human body. Taking advantage of their physical form, various applications for wearable robots are being considered. This study proposes a wearable robot worn on the abdomen and a new interaction with it. Our robot enables a variety of applications related to communication between the wearer and surrounding humans through visual and tactile feedback. The contributions of this research will be (1) the proposal of a novel wearable robot worn on the abdomen and (2) a new interaction with it.
Simultaneous Presentation of Thermal and Mechanical Stimulation Using High-Intensity Airborne Ultrasound
Venue: arXiv cs.HC 2024
Year: 2024

In this study, we propose a non-contact thermal presentation method using airborne ultrasound. We generate strong sound field directly on the human skin and present a perceivable temperature rise. The proposed method enables simultaneous presentation of mechanical and thermal stimuli. In preliminary experiments, we confirmed that temperature increase of 5.4 ${}^\circ$C occurs at the palm after 5.0 s.
Location-Based Output Adaptation for Enhanced Actuator Performance using Frequency Sweep Analysis
Venue: arXiv cs.HC 2024
Year: 2024

This paper presents a methodology for enhancing actuator performance in older devices or retrofitting devices with haptic feedback actuators. The approach is versatile, accommodating various actuator and mounting positions. Through a frequency sweep analysis, the system's characteristics are captured, enabling the creation of location-specific transfer functions to accurately transform input signals into command signals for a precise output at the target location. This method offers fast and simple collection of the system properties and generation of location-specific signals.
Co-Located Magnetic Levitation Haptic and Graphic Display using Iron Core Coils under Screen
Venue: arXiv cs.HC 2024
Year: 2024

This paper describes a combined haptic and graphical interactive system in which a grasped handle is levitated and controlled so that its dynamic rigid-body motion and the forces and torques generated upon it match those of a tool in a real-time simulated environment, displayed on a thin screen on top of the levitation coils and underneath the levitated handle. In this augmented reality configuration, the haptic sensations delivered to the hand of the user and the displayed simulation graphics are perceived in the same location, and the graphical display of the tool acts as a virtual extension of the grasped handle into the displayed simulated environment. The novelty of the system is that it combines iron core levitation coils with a low-cost position sensing system and co-located display in a portable system. The high closed-loop control bandwidth and precise position sensing of the system enable interactive simulated environments to be presented with a convincing degree of realism. The interactive environments to be demonstrated will include 3D rigid-body dynamics, surface contacts with stiffness and damping, and surface texture and friction.
An emotional expression system with vibrotactile feedback during the robot's speech
Venue: arXiv cs.HC 2024
Year: 2024

This study aimed to develop a system that provides vibrotactile feedback corresponding to the emotional content of text when a communication robot speaks. We used OpenAI's "GPT-4o Mini" for emotion estimation, extracting valence and arousal values from the text. The amplitude and frequency of vibrotactile stimulation using sine waves were controlled on the basis of estimated emotional values. We assembled a palm-sized tactile display to present these vibrotactile stimuli. In the experiment, participants listened to the robot's speech while holding the device and then evaluated their psychological state. The results suggested that the communication accompanied by the vibrotactile feedback could influence psychological states and intimacy levels.
Exploiting the Structure of Two Graphs with Graph Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

Graph neural networks (GNNs) have emerged as a promising solution to deal with unstructured data, outperforming traditional deep learning architectures. However, most of the current GNN models are designed to work with a single graph, which limits their applicability in many real-world scenarios where multiple graphs may be involved. To address this limitation, we propose a novel graph-based deep learning architecture to handle tasks where two sets of signals exist, each defined on a different graph. First we consider the setting where the input is represented as a signal on top of one graph (input graph) and the output is a graph signal defined over a different graph (output graph). For this setup, we propose a three-block architecture where we first process the input data using a GNN that operates over the input graph, then apply a transformation function that operates in a latent space and maps the signals from the input to the output graph, and finally implement a second GNN that operates over the output graph. Our goal is not to propose a single specific definition for each of the three blocks, but rather to provide a flexible approach to solve tasks involving data defined on two graphs. The second part of the paper addresses a self-supervised setup, where the focus is not on the output space but on the underlying latent space and, inspired by Canonical Correlation Analysis, we seek informative representations of the data that can be leveraged to solve a downstream task. By leveraging information from multiple graphs, the proposed architecture can capture more intricate relationships between different entities in the data. We test this in several experimental setups using synthetic and real world datasets, and observe that the proposed architecture works better than traditional deep learning architectures, showcasing the importance of leveraging the information of the two graphs.
Development and evaluation of a system to express a sense of telekinesis in VR
Venue: arXiv cs.HC 2024
Year: 2024

Telekinesis is the ability to manipulate remote objects without direct physical contact. In fictional works, telekinesis users are often depicted as controlling objects with their hands and other body parts as if by will alone. Such depictions suggest that users experience a sense of agency over the object despite not physically touching it. In this study, we developed a VR method to simulate telekinesis and investigated whether it is possible to achieve a sense of physical sensation and agency similar to the experience portrayed in fiction.
Socially Assistive Robots: A Technological Approach to Emotional Support
Venue: arXiv cs.RO 2024
Year: 2024

In today's high-pressure and isolated society, the demand for emotional support has surged, necessitating innovative solutions. Socially Assistive Robots (SARs) offer a technological approach to providing emotional assistance by leveraging advanced robotics, artificial intelligence, and sensor technologies. This study explores the development of an emotional support robot designed to detect and respond to human emotions, particularly sadness, through facial recognition and gesture analysis. Utilising the Lego Mindstorms Robotic Kit, Raspberry Pi 4, and various Python libraries, the robot is capable of delivering empathetic interactions, including comforting hugs and AI-generated conversations. Experimental findings highlight the robot's effective facial recognition accuracy, user interaction, and hug feedback mechanisms. These results demonstrate the feasibility of using SARs for emotional support, showcasing their potential features and functions. This research underscores the promise of SARs in providing innovative emotional assistance and enhancing human-robot interaction.
Vibrotactile Feedback for a Remote Operated Robot with Noise Subtraction Based on Perceived Intensity
Venue: arXiv cs.RO 2024
Year: 2024

There is a growing demand for teleoperated robots. This paper presents a novel method for reducing vibration noise generated by robot's own motion, which can disrupt the quality of tactile feedback for teleoperated robots. Our approach focuses on perceived intensity, the amount of how humans experience vibration, to create a noise filter that aligns with human perceptual characteristics. This system effectively subtracts ego-noise while preserving the essential tactile signals, ensuring more accurate and reliable haptic feedback for operators. This method offers a refined solution to the challenge of maintaining high-quality tactile feedback in teleoperated systems.
Magical Experience with Full-body Action
Venue: arXiv cs.HC 2024
Year: 2024

This paper presents a system that generates a magical experience with full-body motion. The system consists of a locomotion interface and a spatial immersive display. A virtual experience system named the Magical Experience Generator was developed, equipped with a Magical Experience Controller. This system provides a physical movement experience along with magical-like interactions in a virtual space. We developed content inspired by the Japanese story "The Man Who Made Flowers Bloom" using Unity as the system's environment. The locomotion interface records the participant's walking trajectory and hand movements, representing their actions in the virtual space.
Effects of Coordinative Arm Swing Movements on the Sense of Agency in Walking Sensation Induced by Kinesthetic Illusion
Venue: arXiv cs.HC 2024
Year: 2024

Kinesthetic illusion can present a sense of movement without actual physical movement of the body, but it often lacks a sense of agency over the movement. Therefore, we focused on the sensation of walking induced by the kinesthetic illusion and hypothesized that incorporating coordinated arm swing movements as actual actions could enhance the sense of agency over the kinesthetic illusion. In this study, we implemented a system that switches the vibrations of the thighs and ankles back and forth based on arm swing movements and investigated whether the sense of agency over the walking sensation induced by the kinesthetic illusion changes with or without arm swing movements. The results suggest a tendency for the sense of agency to be enhanced when arm swing movements are combined.
v-Relax: Virtual Footbath Experiencing by Airflow and Thermal Presentation
Venue: arXiv cs.HC 2024
Year: 2024

Relaxation is a critical counterbalance to the demands of modern business life. Footbaths, a simple yet highly effective therapeutic practice, have been used for centuries across various cultures to promote relaxation and overall well-being. This study presents a novel approach to simulating the experience of a public footbath through the use of tactile and thermal stimulation of airflow to the calf and those on the foot soles. Our system aims to offer a realistic and immersive virtual footbath experience without the need for actual water, by controlling the temperature and airflow to mimic the sensation of soaking feet in water or a water wave. Without using actual water, our system can be more compact, highly responsive, and more reproducible. The layer of airflow is made as thin as possible by adjusting air outlet, and the Coanda effect is also considered to generate a water surface more realistic. The system can provide a multi-sensory experience, including visual and audio feedback of water flow, enhancing the relaxation and therapeutic benefits of a footbath.
A Soft Vibrotactile Display Using Sound Speakers
Venue: arXiv cs.HC 2024
Year: 2024

This study introduces an innovative vibrotactile display that harnesses audio speakers to convey tactile information to the fingertips while preserving the display's softness and flexibility. Our proposed system integrates a flexible polymer body with silicone rubber tubes connected to audio speakers. By streaming audio through these speakers, we induce air vibrations within the tubes, generating tactile stimuli on the skin. In contrast to conventional tactile displays that often rely on bulky, rigid actuators, our approach employs multiple speakers to deliver high-resolution vibration patterns. This configuration enables the presentation of high-frequency vibrations, potentially enhancing the fidelity of tactile feedback. We present a detailed description of the display's design principles and implementation methodology, highlighting its potential to advance the field of haptic interfaces.
Haptic VR Simulation for Surgery Procedures in Medical Training
Venue: arXiv cs.HC 2024
Year: 2024

Traditional medical training faces challenges like ethical concerns, safety risks, and high costs. VR technology offers a promising solution but is limited by low complexity and lack of tactile feedback. This paper presents a cost-effective haptic VR surgery simulation which simulates realistic Kidney Transplant using commercial devices to enhance training authenticity and immersion. Trainees can conduct incision and anastomosis procedures using a haptic stylus device that provides tactile sensations. Results from the test with medical participants showed that haptic feedback positively enhances the VR medical training experience.
Electrostatic Tactile Display without Insulating Layer
Venue: arXiv cs.HC 2024
Year: 2024

This paper explores an approach to eliminating the surface insulating layer in electrostatic (electroadhesion) tactile displays. Electrostatic tactile displays modulate the surface friction by an electrical charge between the skin and the display. Traditionally, the non-conductive dielectric layer has been considered crucial for charge accumulation, as well as for safety to prevent DC current stimulation. However, by utilizing a current control technology for electrotactile displays, we can achieve electrostatic tactile display without the insulating layer. The electrical charge is possibly accumulated in the skin itself or in the air gap between the skin and the electrodes. Safety is maintained by balancing positive and negative current pulses. Furthermore, this system is compatible with existing electrotactile displays. This paper details the system configuration, presentation algorithm, and experimental results. The preliminary trial revealed that five out of eight participants could clearly feel the vibration, confirmed by acceleration recording, while the remaining participants could not experience the sensation.
TelEdge: Haptic Tele-Communication of a Smartphone by Electro-Tactile Stimulation Through the Edges
Venue: arXiv cs.HC 2024
Year: 2024

We present TelEdge, a novel method of remote haptic communication using electrical stimulation through the edges of the smartphone. The aim of this study is to explore communications that can be created by adding touch sensing and haptic feedback using the electrical edge display to conventional audio-visual functionality. We conducted monitoring observations and interviews during a video call between two people, presenting interactive haptic feedback.
HeatFlicker: A Virtual Campfire System Utilizing Flickering Thermal Illusions by Asymmetric Vibrations
Venue: arXiv cs.HC 2024
Year: 2024

In recent years, thermal feedback has emerged as a significant sensory modality in virtual reality. However, the concept of conveying the sensation of thermal movement remains largely unexplored. We propose HeatFlicker, a virtual campfire device that recreates the flickering of fire by using a thermal illusion of moving heat identified in preliminary experiments. This device creates the illusion of heat moving from a fixed heat source. In our demonstration, we provide a novel thermal experience by simulating the flickering of a real fire.