OpenRaiser/Intern-Atlas huggingface dataset:
SimSiam Naming Game: A Unified Approach for Representation Learning and Emergent Communication
Venue: arXiv cs.CL 2024
Year: 2024

Emergent communication, driven by generative models, enables agents to develop a shared language for describing their individual views of the same objects through interactions. Meanwhile, self-supervised learning (SSL), particularly SimSiam, uses discriminative representation learning to make representations of augmented views of the same data point closer in the representation space. Building on the prior work of VI-SimSiam, which incorporates a generative and Bayesian perspective into the SimSiam framework via variational inference (VI) interpretation, we propose SimSiam+VAE, a unified approach for both representation learning and emergent communication. SimSiam+VAE integrates a variational autoencoder (VAE) into the predictor of the SimSiam network to enhance representation learning and capture uncertainty. Experimental results show that SimSiam+VAE outperforms both SimSiam and VI-SimSiam. We further extend this model into a communication framework called the SimSiam Naming Game (SSNG), which applies the generative and Bayesian approach based on VI to develop internal representations and emergent language, while utilizing the discriminative process of SimSiam to facilitate mutual understanding between agents. In experiments with established models, despite the dynamic alternation of agent roles during interactions, SSNG demonstrates comparable performance to the referential game and slightly outperforms the Metropolis-Hastings naming game.
Cross-Domain Transfer Learning Method for Thermal Adaptive Behavior Recognition with WiFi
Venue: arXiv cs.HC 2024
Year: 2024

A reliable comfort model is essential to improve occupant satisfaction and reduce building energy consumption. As two types of the most common and intuitive thermal adaptive behaviors, precise recognition of dressing and undressing can effectively support thermal comfort prediction. However, traditional activity recognition suffers from shortcomings in privacy, cost, and performance. To address the above issues, this study proposes a cross-domain transfer learning method for human dressing and undressing adaptive behavior recognition with WiFi. First, we determine the activity interval by calculating the sliding variance for denoised WiFi signals. Subsequently, short-time Fourier transform and discrete wavelet transform are performed to extract action information on the basis of time-frequency analysis. Ultimately, an efficient 1D CNN pre-trained model is integrated with the SVM algorithm as a hybrid model to enhance the identification robustness in new scenarios. Experiment results show that the hybrid model based on transfer learning provides a more accurate prediction for the adaptative behavior of target subjects, achieving 96.9% and 94.9% accuracy in two cases, respectively.
Accelerated Relaxation Engines for Optimizing to Minimum Energy Path
Venue: arXiv cs.CE 2024
Year: 2024

In the last few decades, several novel algorithms have been designed for finding critical points on PES and the minimum energy paths connecting them. This has led to considerably improve our understanding of reaction mechanisms and kinetics of the underlying processes. These methods implicitly rely on computation of energy and forces on the PES, which are usually obtained by computationally demanding wave-function or density-function based ab initio methods. To mitigate the computational cost, efficient optimization algorithms are needed. Herein, we present two new optimization algorithms: adaptively accelerated relaxation engine (AARE), an enhanced molecular dynamics (MD) scheme, and accelerated conjugate-gradient method (Acc-CG), an improved version of the traditional conjugate gradient (CG) algorithm. We show the efficacy of these algorithms for unconstrained optimization on 2D and 4D test functions. Additionally, we also show the efficacy of these algorithms for optimizing an elastic band of images to the minimum energy path on two analytical potentials (LEPS-I and LEPS-II) and for HCN/CNH isomerization reaction. In all cases, we find that the new algorithms outperforms the standard and popular fast inertial relaxation engine (FIRE).
Advancing Efficient Brain Tumor Multi-Class Classification -- New Insights from the Vision Mamba Model in Transfer Learning
Venue: arXiv cs.CV 2024
Year: 2024

Early and accurate diagnosis of brain tumors is crucial for improving patient survival rates. However, the detection and classification of brain tumors are challenging due to their diverse types and complex morphological characteristics. This study investigates the application of pre-trained models for brain tumor classification, with a particular focus on deploying the Mamba model. We fine-tuned several mainstream transfer learning models and applied them to the multi-class classification of brain tumors. By comparing these models to those trained from scratch, we demonstrated the significant advantages of transfer learning, especially in the medical imaging field, where annotated data is often limited. Notably, we introduced the Vision Mamba (Vim), a novel network architecture, and applied it for the first time in brain tumor classification, achieving exceptional classification accuracy. Experimental results indicate that the Vim model achieved 100% classification accuracy on an independent test set, emphasizing its potential for tumor classification tasks. These findings underscore the effectiveness of transfer learning in brain tumor classification and reveal that, compared to existing state-of-the-art models, the Vim model is lightweight, efficient, and highly accurate, offering a new perspective for clinical applications. Furthermore, the framework proposed in this study for brain tumor classification, based on transfer learning and the Vision Mamba model, is broadly applicable to other medical imaging classification problems.
Bayesian Optimization for Hyperparameters Tuning in Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

This study investigates the application of Bayesian Optimization (BO) for the hyperparameter tuning of neural networks, specifically targeting the enhancement of Convolutional Neural Networks (CNN) for image classification tasks. Bayesian Optimization is a derivative-free global optimization method suitable for expensive black-box functions with continuous inputs and limited evaluation budgets. The BO algorithm leverages Gaussian Process regression and acquisition functions like Upper Confidence Bound (UCB) and Expected Improvement (EI) to identify optimal configurations effectively. Using the Ax and BOTorch frameworks, this work demonstrates the efficiency of BO in reducing the number of hyperparameter tuning trials while achieving competitive model performance. Experimental outcomes reveal that BO effectively balances exploration and exploitation, converging rapidly towards optimal settings for CNN architectures. This approach underlines the potential of BO in automating neural network tuning, contributing to improved accuracy and computational efficiency in machine learning pipelines.
A Longitudinal Analysis of Racial and Gender Bias in New York Times and Fox News Images and Articles
Venue: arXiv cs.CY 2024
Year: 2024

The manner in which different racial and gender groups are portrayed in news coverage plays a large role in shaping public opinion. As such, understanding how such groups are portrayed in news media is of notable societal value, and has thus been a significant endeavour in both the computer and social sciences. Yet, the literature still lacks a longitudinal study examining both the frequency of appearance of different racial and gender groups in online news articles, as well as the context in which such groups are discussed. To fill this gap, we propose two machine learning classifiers to detect the race and age of a given subject. Next, we compile a dataset of 123,337 images and 441,321 online news articles from New York Times (NYT) and Fox News (Fox), and examine representation through two computational approaches. Firstly, we examine the frequency and prominence of appearance of racial and gender groups in images embedded in news articles, revealing that racial and gender minorities are largely under-represented, and when they do appear, they are featured less prominently compared to majority groups. Furthermore, we find that NYT largely features more images of racial minority groups compared to Fox. Secondly, we examine both the frequency and context with which racial minority groups are presented in article text. This reveals the narrow scope in which certain racial groups are covered and the frequency with which different groups are presented as victims and/or perpetrators in a given conflict. Taken together, our analysis contributes to the literature by providing two novel open-source classifiers to detect race and age from images, and shedding light on the racial and gender biases in news articles from venues on opposite ends of the American political spectrum.
Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution
Venue: arXiv cs.LG 2024
Year: 2024

With Deep Reinforcement Learning (DRL) being increasingly considered for the control of real-world systems, the lack of transparency of the neural network at the core of RL becomes a concern. Programmatic Reinforcement Learning (PRL) is able to to create representations of this black-box in the form of source code, not only increasing the explainability of the controller but also allowing for user adaptations. However, these methods focus on distilling a black-box policy into a program and do so after learning using the Mean Squared Error between produced and wanted behaviour, discarding other elements of the RL algorithm. The distilled policy may therefore perform significantly worse than the black-box learned policy. In this paper, we propose to directly learn a program as the policy of an RL agent. We build on TD3 and use its critics as the basis of the objective function of a genetic algorithm that syntheses the program. Our approach builds the program during training, as opposed to after the fact. This steers the program to actual high rewards, instead of a simple Mean Squared Error. Also, our approach leverages the TD3 critics to achieve high sample-efficiency, as opposed to pure genetic methods that rely on Monte-Carlo evaluations. Our experiments demonstrate the validity, explainability and sample-efficiency of our approach in a simple gridworld environment.
Evaluating Perceptual Deviations in Video See-Through Head-Mounted Displays while Utilizing Physical Touchscreens
Venue: arXiv cs.HC 2024
Year: 2024

Extended reality technology has become a useful tool in many applications, but still suffers from visual deviations that can hamper the utility of the technology. This paper discusses the types of persisting visual deviations experienced when observing the natural world through video see-through head-mounted displays. A generalizable method to measure the effect of these deviations on real-world interaction is designed and used in a human-in-the-loop experiment. The experiment compared video see-through sight through an head-mounted display with normal eyesight in a static set-up, focusing on (camera) lens distortions and display deviations. Participants interacted with a real touchscreen, locating the position of flashed markers shortly after disappearance comparing both conditions to check for deviations in position and time. Results show significant larger mean distance errors between the interaction locations and the original marker positions for video see-through compared to normal eyesight. Moreover, errors increase towards the screen periphery. No significant distance error improvement over time was found, however, response times did significantly decrease for both types of sight.
Automated Vulnerability Detection Using Deep Learning Technique
Venue: arXiv cs.CR 2024
Year: 2024

Our work explores the utilization of deep learning, specifically leveraging the CodeBERT model, to enhance code security testing for Python applications by detecting SQL injection vulnerabilities. Unlike traditional security testing methods that may be slow and error-prone, our approach transforms source code into vector representations and trains a Long Short-Term Memory (LSTM) model to identify vulnerable patterns. When compared with existing static application security testing (SAST) tools, our model displays superior performance, achieving higher precision, recall, and F1-score. The study demonstrates that deep learning techniques, particularly with CodeBERT's advanced contextual understanding, can significantly improve vulnerability detection, presenting a scalable methodology applicable to various programming languages and vulnerability types.
Aerodynamic Study of Leading-Edge Protuberance to Improve the Performance of NACA 0009 Blade
Venue: arXiv cs.CE 2024
Year: 2024

Symmetric NACA airfoils tend to undergo abrupt stall characteristics at higher angle of attacks. The abrupt stall has deteriorating effect on lift as well as the efficiency of the airfoils. Abruptness in stall restricts the airfoil to operate only at lower angle of attacks. So, in order to improve the efficiency of airfoils at higher angle of attacks and make it suitable for operation over higher range of angle of attacks, there are many flow control techniques. One such technique is addition of leading-edge protuberance. Leading-edge protuberances are the leading-edge modification of the wing. Leading-edge of the wing is modified with sinusoidal structural modification. This modification has two parameters i.e., Pitch and Amplitude. Many configurations of the protuberances can be obtained by changing the Pitch to Amplitude ratio of the protuberance. In the present work, the Reynolds number is 50k for NACA 0009. The Pitch to Amplitude ratio is varied from PAR1 to PAR27. PAR6 is found to be the better case which has higher lift and efficiency in the post-stall angle of attacks. At the deep stalling AOA of the baseline, i.e., at 13.6o, PAR6 is found to have the highest increase in lift and efficiency compared to the other post stalling AOAs with it having around 39.6% more lift and 27.3% more efficiency compared to the baseline.
Are VLMs Really Blind
Venue: arXiv cs.CL 2024
Year: 2024

Vision Language Models excel in handling a wide range of complex tasks, including Optical Character Recognition (OCR), Visual Question Answering (VQA), and advanced geometric reasoning. However, these models fail to perform well on low-level basic visual tasks which are especially easy for humans. Our goal in this work was to determine if these models are truly "blind" to geometric reasoning or if there are ways to enhance their capabilities in this area. Our work presents a novel automatic pipeline designed to extract key information from images in response to specific questions. Instead of just relying on direct VQA, we use question-derived keywords to create a caption that highlights important details in the image related to the question. This caption is then used by a language model to provide a precise answer to the question without requiring external fine-tuning.
On the Synthesis of Reactive Collision-Free Whole-Body Robot Motions: A Complementarity-based Approach
Venue: arXiv cs.RO 2024
Year: 2024

This paper is about generating motion plans for high degree-of-freedom systems that account for collisions along the entire body. A particular class of mathematical programs with complementarity constraints become useful in this regard. Optimization-based planners can tackle confined-space trajectory planning while being cognizant of robot constraints. However, introducing obstacles in this setting transforms the formulation into a non-convex problem (oftentimes with ill-posed bilinear constraints), which is non-trivial in a real-time setting. To this end, we present the FLIQC (Fast LInear Quadratic Complementarity based) motion planner. Our planner employs a novel motion model that captures the entire rigid robot as well as the obstacle geometry and ensures non-penetration between the surfaces due to the imposed constraint. We perform thorough comparative studies with the state-of-the-art, which demonstrate improved performance. Extensive simulation and hardware experiments validate our claim of generating continuous and reactive motion plans at 1 kHz for modern collaborative robots with constant minimal parameters.
Sing it, Narrate it: Quality Musical Lyrics Translation
Venue: arXiv cs.CL 2024
Year: 2024

Translating lyrics for musicals presents unique challenges due to the need to ensure high translation quality while adhering to singability requirements such as length and rhyme. Existing song translation approaches often prioritize these singability constraints at the expense of translation quality, which is crucial for musicals. This paper aims to enhance translation quality while maintaining key singability features. Our method consists of three main components. First, we create a dataset to train reward models for the automatic evaluation of translation quality. Second, to enhance both singability and translation quality, we implement a two-stage training process with filtering techniques. Finally, we introduce an inference-time optimization framework for translating entire songs. Extensive experiments, including both automatic and human evaluations, demonstrate significant improvements over baseline methods and validate the effectiveness of each component in our approach.
A New Broadcast Primitive for BFT Protocols
Venue: arXiv cs.NI 2024
Year: 2024

Byzantine fault tolerant (BFT) protocol descriptions often assume application-layer networking primitives, such as best-effort and reliable broadcast, which are impossible to implement in practice in a Byzantine environment as they require either unbounded buffering of messages or giving up liveness, under certain circumstances. However, many of these protocols do not (or can be modified to not) need such strong networking primitives. In this paper, we define a new, slightly weaker networking primitive that we call abortable broadcast. We describe an implementation of this new primitive and show that it (1) still provides strong delivery guarantees, even in the case of network congestion, link or peer failure, and backpressure, (2) preserves bandwidth, and (3) enforces all data structures to be bounded even in the presence of malicious peers. The latter prevents out-of-memory DoS attacks by malicious peers, an issue often overlooked in the literature. The new primitive and its implementation are not just theoretical. We use them to implement the BFT protocols in the IC (Internet Computer), a publicly available blockchain network that enables replicated execution of general-purpose computation, serving hundreds of thousands of applications and their users.
4D-based Robot Navigation Using Relativistic Image Processing
Venue: arXiv cs.RO 2024
Year: 2024

Machine perception is an important prerequisite for safe interaction and locomotion in dynamic environments. This requires not only the timely perception of surrounding geometries and distances but also the ability to react to changing situations through predefined, learned but also reusable skill endings of a robot so that physical damage or bodily harm can be avoided. In this context, 4D perception offers the possibility of predicting one's own position and changes in the environment over time. In this paper, we present a 4D-based approach to robot navigation using relativistic image processing. Relativistic image processing handles the temporal-related sensor information in a tensor model within a constructive 4D space. 4D-based navigation expands the causal understanding and the resulting interaction radius of a robot through the use of visual and sensory 4D information.
MStableChain: Towards Multi-Native Stablecoins in EVM-Compatible Blockchain for Stable Fee and Mass Adoption
Venue: arXiv cs.CE 2024
Year: 2024

Traditional blockchain systems, such as Ethereum, typically rely on a \emph{single volatile cryptocurrency for transaction fees}. This leads to fluctuating transaction fee prices and limits the flexibility of users' payment options. To address these issues, we propose MStableChain, which leverage multiple stablecoins as native tokens for transaction fee settlements, thus ensuring stable transaction fees and flexible payment options. To address the challenges of mass adoption and practicality, we propose several core designs. To maintain compatibility with the Ethereum Virtual Machine (EVM) for mass adoption while supporting multiple native stablecoins, MStableChain employs a multi-currency units, multi-type RPCs mechanism. This mechanism enables the system to handle multiple stablecoins without altering the EVM or requiring changes to user applications. Furthermore, an oracle-based gas fee adjustment mechanism is proposed to manage exchange rates between different stablecoins, ensuring equitable transaction costs across various currencies. The system also introduces a secure, on-chain voting-based management protocol for the administrative functions related to these stablecoins. Experimental results from a prototype implementation demonstrate that MStableChain provides stable transaction fee prices, high effectiveness, and good usability.
Hyperspectral Imaging-Based Perception in Autonomous Driving Scenarios: Benchmarking Baseline Semantic Segmentation Models
Venue: arXiv cs.CV 2024
Year: 2024

Hyperspectral Imaging (HSI) is known for its advantages over traditional RGB imaging in remote sensing, agriculture, and medicine. Recently, it has gained attention for enhancing Advanced Driving Assistance Systems (ADAS) perception. Several HSI datasets such as HyKo, HSI-Drive, HSI-Road, and Hyperspectral City have been made available. However, a comprehensive evaluation of semantic segmentation models (SSM) using these datasets is lacking. To address this gap, we evaluated the available annotated HSI datasets on four deep learning-based baseline SSMs: DeepLab v3+, HRNet, PSPNet, and U-Net, along with its two variants: Coordinate Attention (UNet-CA) and Convolutional Block-Attention Module (UNet-CBAM). The original model architectures were adapted to handle the varying spatial and spectral dimensions of the datasets. These baseline SSMs were trained using a class-weighted loss function for individual HSI datasets and evaluated using mean-based metrics such as intersection over union (IoU), recall, precision, F1 score, specificity, and accuracy. Our results indicate that UNet-CBAM, which extracts channel-wise features, outperforms other SSMs and shows potential to leverage spectral information for enhanced semantic segmentation. This study establishes a baseline SSM benchmark on available annotated datasets for future evaluation of HSI-based ADAS perception. However, limitations of current HSI datasets, such as limited dataset size, high class imbalance, and lack of fine-grained annotations, remain significant constraints for developing robust SSMs for ADAS applications.