OpenRaiser/Intern-Atlas huggingface dataset:
Effects of Virtual Hand Representation on the Typing Performance, Upper Extremity Angle, and Neck Muscle Activity during Virtual Reality Typing
Venue: arXiv cs.HC 2024
Year: 2024

This study evaluated the effect of virtual hand representation on the typing performance, upper extremity angle, neck muscle activity, and usability during virtual reality (VR) typing. A total of 15 participants (7 females and 8 males) performed a typing task with and without virtual hand representations. The optical motion capture data was utilized to capture the upper extremity angles, and electromyography device was used to collect the neck muscle activities (left and right splenius capitis). The results showed that the typing performance, upper extremity angle, neck muscle activity, and usability were significantly different with and without virtual hand representations. With the virtual hand representation, net typing speed (WPM) and usability increased significantly by 171.4% and 25% compared to the without hand representation. Without the virtual hand representation, participants showed increased wrist extension, and decreased right shoulder abduction angles. The variability of the neck rotation was increased while typing without the virtual hand representation. The neck muscle activities were increased with the virtual hand representation. The results suggest that typing with the virtual hand representation could positively affect the typing performance and usability and reduce the risk of the musculoskeletal disorders of the upper extremity. Future study could explore the effect of the virtual hand representation for users with varying levels of typing skills.
Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination
Venue: arXiv cs.AI 2024
Year: 2024

The potential of Vision-Language Models (VLMs) often remains underutilized in handling complex text-based problems, particularly when these problems could benefit from visual representation. Resonating with humans' ability to solve complex text-based problems by (1) creating a visual diagram from the problem and (2) deducing what steps they need to take to solve it, we propose Self-Imagine. We leverage a single Vision-Language Model (VLM) to generate a structured representation of the question using HTML, then render the HTML as an image, and finally use the same VLM to answer the question using both the question and the image. Our approach does not require any additional training data or training. We evaluate our approach on three mathematics tasks and nine general-purpose reasoning tasks using state-of-the-art (LLAVA-1.5 and GEMINI PRO) VLMs. Our approach boosts the performance of LLAVA-1.5 and GEMINI PRO on all math tasks (on average GSM8K: +3.1%; ASDIV: +3.2%; SVAMP: +6.9%) and the majority of the general-purpose reasoning tasks by 3.2% to 6.0% on average.
Representation Learning on Event Stream via an Elastic Net-incorporated Tensor Network
Venue: arXiv cs.CV 2024
Year: 2024

Event cameras are neuromorphic sensors that capture asynchronous and sparse event stream when per-pixel brightness changes. The state-of-the-art processing methods for event signals typically aggregate events into a frame or a grid. However, events are dense in time, these works are limited to local information of events due to the stacking. In this paper, we present a novel spatiotemporal representation learning method which can capture the global correlations of all events in the event stream simultaneously by tensor decomposition. In addition, with the events are sparse in space, we propose an Elastic Net-incorporated tensor network (ENTN) model to obtain more spatial and temporal details about event stream. Empirically, the results indicate that our method can represent the spatiotemporal correlation of events with high quality, and can achieve effective results in applications like filtering noise compared with the state-of-the-art methods.
Xaminer: An Internet Cross-Layer Resilience Analysis Tool
Venue: arXiv cs.NI 2024
Year: 2024

A resilient Internet infrastructure is critical in our highly interconnected society. However, the Internet faces several vulnerabilities, ranging from natural disasters to human activities, that can impact the physical layer and, in turn, the higher network layers, such as IP links. In this paper, we introduce Xaminer, the first Internet cross-layer resilience analysis tool, to evaluate the interplay between physical- and network-layer failures. Using a cross-layer Internet map and a failure event model, Xaminer generates a risk profile encompassing a cross-layer impact report, critical infrastructure identification at each layer, and the discovery of trends and patterns under different failure event settings. Xaminer's key strengths lie in its adaptability to diverse disaster scenarios, the ability to assess risks at various granularities, and the capability to generate joint risk profiles for multiple events. We demonstrate Xaminer's capabilities in cross-layer analysis across a spectrum of disaster event models and regions, showcasing its potential role in facilitating well-informed decision-making for resilience planning and deployments.
Transformer-based approach for Ethereum Price Prediction Using Crosscurrency correlation and Sentiment Analysis
Venue: arXiv cs.LG 2024
Year: 2024

The research delves into the capabilities of a transformer-based neural network for Ethereum cryptocurrency price forecasting. The experiment runs around the hypothesis that cryptocurrency prices are strongly correlated with other cryptocurrencies and the sentiments around the cryptocurrency. The model employs a transformer architecture for several setups from single-feature scenarios to complex configurations incorporating volume, sentiment, and correlated cryptocurrency prices. Despite a smaller dataset and less complex architecture, the transformer model surpasses ANN and MLP counterparts on some parameters. The conclusion presents a hypothesis on the illusion of causality in cryptocurrency price movements driven by sentiments.
Adversarial Masking Contrastive Learning for vein recognition
Venue: arXiv cs.CV 2024
Year: 2024

Vein recognition has received increasing attention due to its high security and privacy. Recently, deep neural networks such as Convolutional neural networks (CNN) and Transformers have been introduced for vein recognition and achieved state-of-the-art performance. Despite the recent advances, however, existing solutions for finger-vein feature extraction are still not optimal due to scarce training image samples. To overcome this problem, in this paper, we propose an adversarial masking contrastive learning (AMCL) approach, that generates challenging samples to train a more robust contrastive learning model for the downstream palm-vein recognition task, by alternatively optimizing the encoder in the contrastive learning model and a set of latent variables. First, a huge number of masks are generated to train a robust generative adversarial network (GAN). The trained generator transforms a latent variable from the latent variable space into a mask space. Then, we combine the trained generator with a contrastive learning model to obtain our AMCL, where the generator produces challenging masking images to increase the contrastive loss and the contrastive learning model is trained based on the harder images to learn a more robust feature representation. After training, the trained encoder in the contrastive learning model is combined with a classification layer to build a classifier, which is further fine-tuned on labeled training data for vein recognition. The experimental results on three databases demonstrate that our approach outperforms existing contrastive learning approaches in terms of improving identification accuracy of vein classifiers and achieves state-of-the-art recognition results.
SpecSTG: A Fast Spectral Diffusion Framework for Probabilistic Spatio-Temporal Traffic Forecasting
Venue: arXiv cs.LG 2024
Year: 2024

Traffic forecasting, a crucial application of spatio-temporal graph (STG) learning, has traditionally relied on deterministic models for accurate point estimations. Yet, these models fall short of quantifying future uncertainties. Recently, many probabilistic methods, especially variants of diffusion models, have been proposed to fill this gap. However, existing diffusion methods typically deal with individual sensors separately when generating future time series, resulting in limited usage of spatial information in the probabilistic learning process. In this work, we propose SpecSTG, a novel spectral diffusion framework, to better leverage spatial dependencies and systematic patterns inherent in traffic data. More specifically, our method generates the Fourier representation of future time series, transforming the learning process into the spectral domain enriched with spatial information. Additionally, our approach incorporates a fast spectral graph convolution designed for Fourier input, alleviating the computational burden associated with existing models. Compared with state-of-the-arts, SpecSTG achieves up to 8% improvements on point estimations and up to 0.78% improvements on quantifying future uncertainties. Furthermore, SpecSTG's training and validation speed is 3.33X of the most efficient existing diffusion method for STG forecasting. The source code for SpecSTG is available at https://anonymous.4open.science/r/SpecSTG.
The Devil is in the Details: Boosting Guided Depth Super-Resolution via Rethinking Cross-Modal Alignment and Aggregation
Venue: arXiv cs.CV 2024
Year: 2024

Guided depth super-resolution (GDSR) involves restoring missing depth details using the high-resolution RGB image of the same scene. Previous approaches have struggled with the heterogeneity and complementarity of the multi-modal inputs, and neglected the issues of modal misalignment, geometrical misalignment, and feature selection. In this study, we rethink some essential components in GDSR networks and propose a simple yet effective Dynamic Dual Alignment and Aggregation network (D2A2). D2A2 mainly consists of 1) a dynamic dual alignment module that adapts to alleviate the modal misalignment via a learnable domain alignment block and geometrically align cross-modal features by learning the offset; and 2) a mask-to-pixel feature aggregate module that uses the gated mechanism and pixel attention to filter out irrelevant texture noise from RGB features and combine the useful features with depth features. By combining the strengths of RGB and depth features while minimizing disturbance introduced by the RGB image, our method with simple reuse and redesign of basic components achieves state-of-the-art performance on multiple benchmark datasets. The code is available at https://github.com/JiangXinni/D2A2.
IoTWarden: A Deep Reinforcement Learning Based Real-time Defense System to Mitigate Trigger-action IoT Attacks
Venue: arXiv cs.CR 2024
Year: 2024

In trigger-action IoT platforms, IoT devices report event conditions to IoT hubs notifying their cyber states and let the hubs invoke actions in other IoT devices based on functional dependencies defined as rules in a rule engine. These functional dependencies create a chain of interactions that help automate network tasks. Adversaries exploit this chain to report fake event conditions to IoT hubs and perform remote injection attacks upon a smart environment to indirectly control targeted IoT devices. Existing defense efforts usually depend on static analysis over IoT apps to develop rule-based anomaly detection mechanisms. We also see ML-based defense mechanisms in the literature that harness physical event fingerprints to determine anomalies in an IoT network. However, these methods often demonstrate long response time and lack of adaptability when facing complicated attacks. In this paper, we propose to build a deep reinforcement learning based real-time defense system for injection attacks. We define the reward functions for defenders and implement a deep Q-network based approach to identify the optimal defense policy. Our experiments show that the proposed mechanism can effectively and accurately identify and defend against injection attacks with reasonable computation overhead.
Channel Estimation for Holographic Communications in Hybrid Near-Far Field
Venue: arXiv cs.IT 2024
Year: 2024

To realize holographic communications, a potential technology for spectrum efficiency improvement in the future sixth-generation (6G) network, antenna arrays inlaid with numerous antenna elements will be deployed. However, the increase in antenna aperture size makes some users lie in the Fresnel region, leading to the hybrid near-field and far-field communication mode, where the conventional far-field channel estimation methods no longer work well. To tackle the above challenge, this paper considers channel estimation in a hybrid-field multipath environment, where each user and each scatterer can be in either the far-field or the near-field region. First, a joint angular-polar domain channel transform is designed to capture the hybrid-field channel's near-field and far-field features. We then analyze the power diffusion effect in the hybrid-field channel, which indicates that the power corresponding to one near-field (far-field) path component of the multipath channel may spread to far-field (near-field) paths and causes estimation error. We design a novel power-diffusion-based orthogonal matching pursuit channel estimation algorithm (PD-OMP). It can eliminate the prior knowledge requirement of path numbers in the far field and near field, which is a must in other OMP-based channel estimation algorithms. Simulation results show that PD-OMP outperforms current hybrid-field channel estimation methods.
Agile Meets Quantum: A Novel Genetic Algorithm Model for Predicting the Success of Quantum Software Development Project
Venue: arXiv cs.SE 2024
Year: 2024

Context: Quantum software systems represent a new realm in software engineering, utilizing quantum bits (Qubits) and quantum gates (Qgates) to solve the complex problems more efficiently than classical counterparts . Agile software development approaches are considered to address many inherent challenges in quantum software development, but their effective integration remains unexplored Objective: This study investigates key causes of challenges that could hinders the adoption of traditional agile approaches in quantum software projects and develop an Agile Quantum Software Project Success Prediction Model (AQSSPM). Methodology: Firstly, w e identified 19 causes of challenging factors discussed in our previous study, which are potentially impacting agile quantum project success. Secondly, a survey was conducted to collect expert opinions on these causes and applied Genetic Algorithm (GA) with Na i ve Bayes Classifier (NBC) and Logistic Regression (LR) to develop the AQSSPM Results: Utilizing GA with NBC, project success probability improved from 53.17% to 99.68%, with cost reductions from 0.463% to 0.403%. Similarly, GA with LR increased success rates from 55.52% to 98.99%, and costs decreased from 0.496% to 0.409% after 100 iterati ons. Both methods result showed a strong positive correlation (rs=0.955) in causes ranking, with no significant difference between them (t=1.195, p=0.240>0.05). Conclusion: The AQSSPM highlights critical focus areas for efficiently and successfully implementing agile quantum projects considering the cost factor of a particular project
TLIC: Learned Image Compression with ROI-Weighted Distortion and Bit Allocation
Venue: arXiv cs.CV 2024
Year: 2024

This short paper describes our method for the track of image compression. To achieve better perceptual quality, we use the adversarial loss to generate realistic textures, use region of interest (ROI) mask to guide the bit allocation for different regions. Our Team name is TLIC.
Near-Far Field Codebook Design for IOS-Aided Multi-User Communications
Venue: arXiv cs.IT 2024
Year: 2024

Recently, the rapid development of metasurface facilitates the growth of extremely large-scale antenna arrays, making the ultra-massive MIMO possible. In this paper, we study the codebook design and beam training for an intelligent omni-surface (IOS) aided multi-user system, where the IOS is a novel metasurface enabling simultaneous signal reflection and refraction. To deal with the near field expansion caused by the large-dimension of IOS, we design a near-far field codebook to serve users both in the near and far fields without prior knowledge of user distribution. Moreover, to fully exploit the dual functionality of the IOS, the coupling between the reflective and refractive signals is analyzed theoretically and utilized in the codebook design, thereby reducing the training overhead. On this basis, the multi-user beam training is adopted where each codeword covers multiple areas to enable all users to be trained simultaneously. Simulation results verify our theoretical analysis on the reflective-refractive coupling. Compared to the state-of-the-art schemes, the proposed scheme can improve the sum rate and throughput.
Key-point Guided Deformable Image Manipulation Using Diffusion Model
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we introduce a Key-point-guided Diffusion probabilistic Model (KDM) that gains precise control over images by manipulating the object's key-point. We propose a two-stage generative model incorporating an optical flow map as an intermediate output. By doing so, a dense pixel-wise understanding of the semantic relation between the image and sparse key point is configured, leading to more realistic image generation. Additionally, the integration of optical flow helps regulate the inter-frame variance of sequential images, demonstrating an authentic sequential image generation. The KDM is evaluated with diverse key-point conditioned image synthesis tasks, including facial image generation, human pose synthesis, and echocardiography video prediction, demonstrating the KDM is proving consistency enhanced and photo-realistic images compared with state-of-the-art models.
LiveScaler: Live control of the harmony of an electronic music track
Venue: arXiv cs.SD 2024
Year: 2024

In Electronic Dance Music (EDM), many artists use DJing techniques in order to perform their own productions live. As a consequence, they do not have access during the performance to the internal structure of their tracks, and specifically to their equivalent of a partition: MIDI files. On the other hand, if an artist attempts to remix or interpret their own production live, the number of tracks that they can simultaneously control is limited without suitable software. This article introduces LiveScaler, a software that allows live control of the harmony and pitch of electronic music. A set of pitch transformations, termed affine transformations, is presented. These transformations are applied to all MIDI streams of a prepared track. A MaxMSP implementation, in conjunction with Ableton Live, is proposed. Special attention is given to control issues, mapping, and practical live experimentation in the context of EDM.
DPAFNet:Dual Path Attention Fusion Network for Single Image Deraining
Venue: arXiv cs.CV 2024
Year: 2024

Rainy weather will have a significant impact on the regular operation of the imaging system. Based on this premise, image rain removal has always been a popular branch of low-level visual tasks, especially methods using deep neural networks. However, most neural networks are but-branched, such as only using convolutional neural networks or Transformers, which is unfavourable for the multidimensional fusion of image features. In order to solve this problem, this paper proposes a dual-branch attention fusion network. Firstly, a two-branch network structure is proposed. Secondly, an attention fusion module is proposed to selectively fuse the features extracted by the two branches rather than simply adding them. Finally, complete ablation experiments and sufficient comparison experiments prove the rationality and effectiveness of the proposed method.
IsamasRed: A Public Dataset Tracking Reddit Discussions on Israel-Hamas Conflict
Venue: arXiv cs.SI 2024
Year: 2024

The conflict between Israel and Palestinians significantly escalated after the October 7, 2023 Hamas attack, capturing global attention. To understand the public discourse on this conflict, we present a meticulously compiled dataset-IsamasRed-comprising nearly 400,000 conversations and over 8 million comments from Reddit, spanning from August 2023 to November 2023. We introduce an innovative keyword extraction framework leveraging a large language model to effectively identify pertinent keywords, ensuring a comprehensive data collection. Our initial analysis on the dataset, examining topics, controversy, emotional and moral language trends over time, highlights the emotionally charged and complex nature of the discourse. This dataset aims to enrich the understanding of online discussions, shedding light on the complex interplay between ideology, sentiment, and community engagement in digital spaces.
ModelNet-O: A Large-Scale Synthetic Dataset for Occlusion-Aware Point Cloud Classification
Venue: arXiv cs.CV 2024
Year: 2024

Recently, 3D point cloud classification has made significant progress with the help of many datasets. However, these datasets do not reflect the incomplete nature of real-world point clouds caused by occlusion, which limits the practical application of current methods. To bridge this gap, we propose ModelNet-O, a large-scale synthetic dataset of 123,041 samples that emulate real-world point clouds with self-occlusion caused by scanning from monocular cameras. ModelNet-O is 10 times larger than existing datasets and offers more challenging cases to evaluate the robustness of existing methods. Our observation on ModelNet-O reveals that well-designed sparse structures can preserve structural information of point clouds under occlusion, motivating us to propose a robust point cloud processing method that leverages a critical point sampling (CPS) strategy in a multi-level manner. We term our method PointMLS. Through extensive experiments, we demonstrate that our PointMLS achieves state-of-the-art results on ModelNet-O and competitive results on regular datasets, and it is robust and effective. More experiments also demonstrate the robustness and effectiveness of PointMLS.
Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication
Venue: arXiv cs.CV 2024
Year: 2024

Leveraging Large Multimodal Models (LMMs) to simulate human behaviors when processing multimodal information, especially in the context of social media, has garnered immense interest due to its broad potential and far-reaching implications. Emojis, as one of the most unique aspects of digital communication, are pivotal in enriching and often clarifying the emotional and tonal dimensions. Yet, there is a notable gap in understanding how these advanced models, such as GPT-4V, interpret and employ emojis in the nuanced context of online interaction. This study intends to bridge this gap by examining the behavior of GPT-4V in replicating human-like use of emojis. The findings reveal a discernible discrepancy between human and GPT-4V behaviors, likely due to the subjective nature of human interpretation and the limitations of GPT-4V's English-centric training, suggesting cultural biases and inadequate representation of non-English cultures.