OpenRaiser/Intern-Atlas huggingface dataset:
Musinger: Communication of Music over a Distance with Wearable Haptic Display and Touch Sensitive Surface
Venue: arXiv cs.HC 2024
Year: 2024

This study explores the integration of auditory and tactile experiences in musical haptics, focusing on enhancing sensory dimensions of music through touch. Addressing the gap in translating auditory signals to meaningful tactile feedback, our research introduces a novel method involving a touch-sensitive recorder and a wearable haptic display that captures musical interactions via force sensors and converts these into tactile sensations. Previous studies have shown the potential of haptic feedback to enhance musical expressivity, yet challenges remain in conveying complex musical nuances. Our method aims to expand music accessibility for individuals with hearing impairments and deepen digital musical interactions. Experimental results reveal high accuracy ($98\%$ without noise, 93% with white noise) in melody recognition through tactile feedback, demonstrating effective transmission and perception of musical information. The findings highlight the potential of haptic technology to bridge sensory gaps, offering significant implications for music therapy, education, and remote musical collaboration, advancing the field of musical haptics and multi-sensory technology applications.
CiteClick: A Browser Extension for Real-Time Scholar Citation Tracking
Venue: arXiv cs.DL 2024
Year: 2024

This technical report presents CiteClick, a browser extension designed to monitor and track Google Scholar citation counts for multiple researchers in real-time. We discuss the motivation behind the tool, its key features, implementation details, and potential impact on the academic community. The report covers installation procedures, usage guidelines, and customization options, concluding with a discussion on future work and potential improvements. By automating the process of citation tracking, CiteClick aims to enhance research evaluation processes and facilitate more informed decision-making in academic contexts.
Managing Bandwidth: The Key to Cloud-Assisted Autonomous Driving
Venue: arXiv cs.NI 2024
Year: 2024

Prevailing wisdom asserts that one cannot rely on the cloud for critical real-time control systems like self-driving cars. We argue that we can, and must. Following the trends of increasing model sizes, improvements in hardware, and evolving mobile networks, we identify an opportunity to offload parts of time-sensitive and latency-critical compute to the cloud. Doing so requires carefully allocating bandwidth to meet strict latency SLOs, while maximizing benefit to the car.
MoRE: Multi-Modal Contrastive Pre-training with Transformers on X-Rays, ECGs, and Diagnostic Report
Venue: arXiv cs.AI 2024
Year: 2024

In this paper, we introduce a novel Multi-Modal Contrastive Pre-training Framework that synergistically combines X-rays, electrocardiograms (ECGs), and radiology/cardiology reports. Our approach leverages transformers to encode these diverse modalities into a unified representation space, aiming to enhance diagnostic accuracy and facilitate comprehensive patient assessments. We utilize LoRA-Peft to significantly reduce trainable parameters in the LLM and incorporate recent linear attention dropping strategy in the Vision Transformer(ViT) for smoother attention. Furthermore, we provide novel multimodal attention explanations and retrieval for our model. To the best of our knowledge, we are the first to propose an integrated model that combines X-ray, ECG, and Radiology/Cardiology Report with this approach. By utilizing contrastive loss, MoRE effectively aligns modality-specific features into a coherent embedding, which supports various downstream tasks such as zero-shot classification and multimodal retrieval. Employing our proposed methodology, we achieve state-of-the-art (SOTA) on the Mimic-IV, CheXpert, Edema Severity, and PtbXl downstream datasets, surpassing existing multimodal approaches. Our proposed framework shows significant improvements in capturing intricate inter-modal relationships and its robustness in medical diagnosis that establishes a framework for future research in multimodal learning in the healthcare sector.
Edge Computing in Distributed Acoustic Sensing: An Application in Traffic Monitoring
Venue: arXiv cs.NI 2024
Year: 2024

Distributed acoustic sensing (DAS) technology leverages fiber optic cables to detect vibrations and acoustic events, which is a promising solution for real-time traffic monitoring. In this paper, we introduce a novel methodology for detecting and tracking vehicles using DAS data, focusing on real-time processing through edge computing. Our approach applies the Hough transform to detect straight-line segments in the spatiotemporal DAS data, corresponding to vehicles crossing the Astfjord bridge in Norway. These segments are further clustered using the Density-based spatial clustering of applications with noise (DBSCAN) algorithm to consolidate multiple detections of the same vehicle, reducing noise and improving accuracy. The proposed workflow effectively counts vehicles and estimates their speed with only tens of seconds latency, enabling real-time traffic monitoring on the edge. To validate the system, we compare DAS data with simultaneous video footage, achieving high accuracy in vehicle detection, including the distinction between cars and trucks based on signal strength and frequency content. Results show that the system is capable of processing large volumes of data efficiently. We also analyze vehicle speeds and traffic patterns, identifying temporal trends and variations in traffic flow. Real-time deployment on edge devices allows immediate analysis and visualization via cloud-based platforms. In addition to traffic monitoring, the method successfully detected structural responses in the bridge, highlighting its potential use in structural health monitoring.
Verification of Neural Control Barrier Functions with Symbolic Derivative Bounds Propagation
Venue: arXiv cs.RO 2024
Year: 2024

Control barrier functions (CBFs) are important in safety-critical systems and robot control applications. Neural networks have been used to parameterize and synthesize CBFs with bounded control input for complex systems. However, it is still challenging to verify pre-trained neural networks CBFs (neural CBFs) in an efficient symbolic manner. To this end, we propose a new efficient verification framework for ReLU-based neural CBFs through symbolic derivative bound propagation by combining the linearly bounded nonlinear dynamic system and the gradient bounds of neural CBFs. Specifically, with Heaviside step function form for derivatives of activation functions, we show that the symbolic bounds can be propagated through the inner product of neural CBF Jacobian and nonlinear system dynamics. Through extensive experiments on different robot dynamics, our results outperform the interval arithmetic based baselines in verified rate and verification time along the CBF boundary, validating the effectiveness and efficiency of the proposed method with different model complexity. The code can be found at https://github.com/intelligent-control-lab/ verify-neural-CBF.
Assessing the Performance of Human-Capable LLMs -- Are LLMs Coming for Your Job?
Venue: arXiv cs.CY 2024
Year: 2024

The current paper presents the development and validation of SelfScore, a novel benchmark designed to assess the performance of automated Large Language Model (LLM) agents on help desk and professional consultation tasks. Given the increasing integration of AI in industries, particularly within customer service, SelfScore fills a crucial gap by enabling the comparison of automated agents and human workers. The benchmark evaluates agents on problem complexity and response helpfulness, ensuring transparency and simplicity in its scoring system. The study also develops automated LLM agents to assess SelfScore and explores the benefits of Retrieval-Augmented Generation (RAG) for domain-specific tasks, demonstrating that automated LLM agents incorporating RAG outperform those without. All automated LLM agents were observed to perform better than the human control group. Given these results, the study raises concerns about the potential displacement of human workers, especially in areas where AI technologies excel. Ultimately, SelfScore provides a foundational tool for understanding the impact of AI in help desk environments while advocating for ethical considerations in the ongoing transition towards automation.
Challenges and Opportunities: Implementing Diversity and Inclusion in Software Engineering University Level Education in Finland
Venue: arXiv cs.CY 2024
Year: 2024

Considerable efforts have been made at the high school level to encourage girls to pursue software engineering careers and raise awareness about diversity within the field. Similarly, software companies have become more active in diversity and inclusion (D&I) topics, aiming to create more inclusive work environments. However, the way diversity and inclusion are approached inside software engineering university education remains less clear. This study investigates the current state of D&I in software engineering education and faculties in Finland. An online survey (N=30) was conducted among Finnish software engineering university teachers to investigate which approaches and case examples of D&I are most commonly used by software engineering teachers in Finland. In addition, it was researched how software engineering teachers perceive the importance of D&I in their courses. As a result of the quantitative and thematic analysis, a framework to identify attitudes, approaches, challenges and pedagogical strategies when implementing D&I themes in software engineering education is presented. This framework also offers a process for integrating D&I themes for the curriculum or at the faculty level. The findings of this study emphasize that there is a continuing need for diverse-aware education and training. The results underline the responsibility of universities to ensure that future professionals are equipped with the necessary skills and knowledge to promote D&I in the field of software engineering.
Accelerating Biological Spatial Cluster Analysis with the Parallel Integral Image Technique
Venue: arXiv cs.CV 2024
Year: 2024

Spatial cluster analysis (SCA) offers valuable insights into biological images; a common SCA technique is sliding window analysis (SWA). Unfortunately, SWA's computational cost hinders its application to larger images, limiting its use to small-scale images. With advancements in high-resolution microscopy, images now exceed the capabilities of previous SWA approaches, reaching sizes up to 70,000 by 85,000 pixels. To overcome these limitations, this paper introduces the parallel integral image approach to SWA, surpassing previous methods. We achieve a remarkable speedup of 131,806x on small-scale images and consistent speedups of over 10,000x on a variety of large-scale microscopy images. We analyze the computational complexity advantages of the parallel integral image approach and present experimental results that validate the superior performance of integral-image-based methods. Our approach is made available as an open-source Python PIP package available at https://github.com/OckermanSethGVSU/BioPII.
Illuminating Networks: Enhancing Visible Light Communication with Physical-Layer Network Coding (PNC-VLC Scheme)
Venue: arXiv cs.IT 2024
Year: 2024

Wireless communications using light waves are called visible light communication. A technique called Physical-layer Network coding allows several users to communicate simultaneously over the same channel in a distributed space-time Coding scheme. Herein, we report a new PNC-VLC one-to-many scheme using physical-layer network coding in VLC system for enhancing throughput by boosting data transmission at relay nodes. We develop an integrated PNC-VLC framework including physical-layer signal processing algorithms and medium access control approaches. Our proposed technique is then modelled mathematically and compared with some of the VLC and PNC-VLC schemes under various channel conditions and scenarios. Our investigation revealed that our proposed technique performed better than traditional PNC schemes in achieving better Bit Error Rate performance in different channel
FlashHack: Reflections on the Usage of a Micro Hackathon as an Assessment Tool in a Machine Learning Course
Venue: arXiv cs.CY 2024
Year: 2024

Machine learning (ML) course for undergraduates face challenges in assessing student learning and providing practical exposure. Group project-based learning, an increasingly popular form of experiential learning in CS education, encounters certain limitation in participation and non-participation from a few students. Studies also suggest that students find longer programming assignments and project-based assessments distracting and struggle to maintain focus when they coincide with other courses. To tackle these issues, we introduced FlashHack: a monitored, incremental, in-classroom micro Hackathon that combines project-based learning with Hackathon elements. Engaging 229 third year CS undergraduate students in teams of four, FlashHack prompted them to tackle predefined challenges using machine learning techniques within a set timeframe. Assessment criteria emphasized machine learning application, problem-solving, collaboration, and creativity. Our results indicate high student engagement and satisfaction, alongside simplified assessment processes for instructors. This experience report outlines the Hackathon design and implementation, highlights successes and areas for improvement making it feasible for replication by interested computing educators.
Magnetic Ball Chain Robots for Cardiac Arrhythmia Treatment
Venue: arXiv cs.RO 2024
Year: 2024

This paper introduces a novel magnetic navigation system for cardiac ablation. The system is formed from two key elements: a magnetic ablation catheter consisting of a chain of spherical permanent magnets; and an actuation system comprised of two cart-mounted permanent magnets undergoing pure rotation. The catheter design enables a large magnetic content with the goal of minimizing the footprint of the actuation system for easier integration with the clinical workflow. We present a quasi-static model of the catheter, the design of the actuation units, and their control modalities. Experimental validation shows that we can use small rotating magnets (119mm diameter) to reach cardiac ablation targets while generating clinically-relevant forces. Catheter control using a joystick is compared with manual catheter control. blue While total task completion time is similar, smoother navigation is observed using the proposed robotic system. We also demonstrate that the ball chain can ablate heart tissue and generate lesions comparable to the current clinical ablation catheters.
AlignVSR: Audio-Visual Cross-Modal Alignment for Visual Speech Recognition
Venue: arXiv cs.SD 2024
Year: 2024

Visual Speech Recognition (VSR) aims to recognize corresponding text by analyzing visual information from lip movements. Due to the high variability and weak information of lip movements, VSR tasks require effectively utilizing any information from any source and at any level. In this paper, we propose a VSR method based on audio-visual cross-modal alignment, named AlignVSR. The method leverages the audio modality as an auxiliary information source and utilizes the global and local correspondence between the audio and visual modalities to improve visual-to-text inference. Specifically, the method first captures global alignment between video and audio through a cross-modal attention mechanism from video frames to a bank of audio units. Then, based on the temporal correspondence between audio and video, a frame-level local alignment loss is introduced to refine the global alignment, improving the utility of the audio information. Experimental results on the LRS2 and CNVSRC.Single datasets consistently show that AlignVSR outperforms several mainstream VSR methods, demonstrating its superior and robust performance.
Secure Computation and Trustless Data Intermediaries in Data Spaces
Venue: arXiv cs.CR 2024
Year: 2024

This paper explores the integration of advanced cryptographic techniques for secure computation in data spaces to enable secure and trusted data sharing, which is essential for the evolving data economy. In addition, the paper examines the role of data intermediaries, as outlined in the EU Data Governance Act, in data spaces and specifically introduces the idea of trustless intermediaries that do not have access to their users' data. Therefore, we exploit the introduced secure computation methods, i.e. Secure Multi-Party Computation (MPC) and Fully Homomorphic Encryption (FHE), and discuss the security benefits. Overall, we identify and address key challenges for integration, focusing on areas such as identity management, policy enforcement, node selection, and access control, and present solutions through real-world use cases, including air traffic management, manufacturing, and secondary data use. Furthermore, through the analysis of practical applications, this work proposes a comprehensive framework for the implementation and standardization of secure computing technologies in dynamic, trustless data environments, paving the way for future research and development of a secure and interoperable data ecosystem.
Susu Box or Piggy Bank: Assessing Cultural Commonsense Knowledge between Ghana and the U.S
Venue: arXiv cs.CL 2024
Year: 2024

Recent work has highlighted the culturally-contingent nature of commonsense knowledge. We introduce AMAMMER$ε$, a test set of 525 multiple-choice questions designed to evaluate the commonsense knowledge of English LLMs, relative to the cultural contexts of Ghana and the United States. To create AMAMMER$ε$, we select a set of multiple-choice questions (MCQs) from existing commonsense datasets and rewrite them in a multi-stage process involving surveys of Ghanaian and U.S. participants. In three rounds of surveys, participants from both pools are solicited to (1) write correct and incorrect answer choices, (2) rate individual answer choices on a 5-point Likert scale, and (3) select the best answer choice from the newly-constructed MCQ items, in a final validation step. By engaging participants at multiple stages, our procedure ensures that participant perspectives are incorporated both in the creation and validation of test items, resulting in high levels of agreement within each pool. We evaluate several off-the-shelf English LLMs on AMAMMER$ε$. Uniformly, models prefer answers choices that align with the preferences of U.S. annotators over Ghanaian annotators. Additionally, when test items specify a cultural context (Ghana or the U.S.), models exhibit some ability to adapt, but performance is consistently better in U.S. contexts than Ghanaian. As large resources are devoted to the advancement of English LLMs, our findings underscore the need for culturally adaptable models and evaluations to meet the needs of diverse English-speaking populations around the world.
Evaluating the Performance of a D-Wave Quantum Annealing System for Feature Subset Selection in Software Defect Prediction
Venue: arXiv cs.SE 2024
Year: 2024

Predicting software defects early in the development process not only enhances the quality and reliability of the software but also decreases the cost of development. A wide range of machine learning techniques can be employed to create software defect prediction models, but the effectiveness and accuracy of these models are often influenced by the choice of appropriate feature subset. Since finding the optimal feature subset is computationally intensive, heuristic and metaheuristic approaches are commonly employed to identify near-optimal solutions within a reasonable time frame. Recently, the quantum computing paradigm quantum annealing (QA) has been deployed to find solutions to complex optimization problems. This opens up the possibility of addressing the feature subset selection problem with a QA machine. Although several strategies have been proposed for feature subset selection using a QA machine, little exploration has been done regarding the viability of a QA machine for feature subset selection in software defect prediction. This study investigates the potential of D-Wave QA system for this task, where we formulate a mutual information (MI)-based filter approach as an optimization problem and utilize a D-Wave Quantum Processing Unit (QPU) solver as a QA solver for feature subset selection. We evaluate the performance of this approach using multiple software defect datasets from the AEEM, JIRA, and NASA projects. We also utilize a D-Wave classical solver for comparative analysis. Our experimental results demonstrate that QA-based feature subset selection can enhance software defect prediction. Although the D-Wave QPU solver exhibits competitive prediction performance with the classical solver in software defect prediction, it significantly reduces the time required to identify the best feature subset compared to its classical counterpart.
In Search of the Successful Interpolation: On the Role of Sharpness in CLIP Generalization
Venue: arXiv cs.LG 2024
Year: 2024

\textit{Zero-shot} models like CLIP are often fine-tuned on a target dataset to improve its accuracy further, but this can compromise out-of-distribution (OOD) robustness. Robust Fine-Tuning (\texttt{RFT} )~\citep{wortsman2021robust}, which interpolates between the \textit{zero-shot} and \textit{fine-tuned} models, has been proposed to address this issue. However, understanding when \texttt{RFT} actually improves OOD error remains limited. In this work, we empirically investigate the robustness of \texttt{RFT} in CLIP models, with a focus on the \textit{sharpness} of the CLIP model during interpolation. First, we demonstrate that while sharpness may not serve as a reliable indicator for predicting the generalization of modern architectures like CLIP on OOD data, this challenges the conventional belief in the generalization benefits of flat minima in foundation models. However, by examining the role of the \textit{straggler layer} phenomenon, we show that, unlike overall sharpness, the \textit{layer-wise} sharpness of \textit{straggler} layers can reliably capture the generalization performance of interpolated CLIP models on OOD data. Our extensive experiments reveal that \textit{layer-wise} sharpness correlates with generalization in OOD accuracy for \texttt{RFT}. Furthermore, we demonstrate that by inducing sparsity in the \textit{straggler} layers, we can mitigate the \textit{failure mode} phenomenon in \texttt{RFT}. To the best of our knowledge, this is the first work to study the role of sharpness in the \textit{success} of interpolation in the weight space of CLIP foundation models. Our code is available at \url{https://github.com/alirezaabdollahpour/CLIP_Mode_Connectivity}.
Survival Multiarmed Bandits with Bootstrapping Methods
Venue: arXiv cs.LG 2024
Year: 2024

The Multiarmed Bandits (MAB) problem has been extensively studied and has seen many practical applications in a variety of fields. The Survival Multiarmed Bandits (S-MAB) open problem is an extension which constrains an agent to a budget that is directly related to observed rewards. As budget depletion leads to ruin, an agent's objective is to both maximize expected cumulative rewards and minimize the probability of ruin. This paper presents a framework that addresses such a dual goal using an objective function balanced by a ruin aversion component. Action values are estimated through a novel approach which consists of bootstrapping samples from previously observed rewards. In numerical experiments, the policies we present outperform benchmarks from the literature.