OpenRaiser/Intern-Atlas huggingface dataset:
Evaluation Framework for Feedback Generation Methods in Skeletal Movement Assessment
Venue: arXiv cs.CV 2024
Year: 2024

The application of machine-learning solutions to movement assessment from skeleton videos has attracted significant research attention in recent years. This advancement has made rehabilitation at home more accessible, utilizing movement assessment algorithms that can operate on affordable equipment for human pose detection and analysis from 2D or 3D videos. While the primary objective of automatic assessment tasks is to score movements, the automatic generation of feedback highlighting key movement issues has the potential to significantly enhance and accelerate the rehabilitation process. While numerous research works exist in the field of automatic movement assessment, only a handful address feedback generation. In this study, we propose terminology and criteria for the classification, evaluation, and comparison of feedback generation solutions. We discuss the challenges associated with each feedback generation approach and use our proposed criteria to classify existing solutions. To our knowledge, this is the first work that formulates feedback generation in skeletal movement assessment.
Data Analysis Methods Preliminaries for a Photon-based Hardware Random Number Generator
Venue: arXiv cs.CR 2024
Year: 2024

High quality random numbers are necessary in the modern world. Ranging from encryption keys in cyber security to models and simulations for scientific use: it's important that these random numbers are of high quality and quickly attainable. One common solution to the generation of random numbers is that of pseudo-random number generators, or PRNGs. PRNGs generate random numbers by first quantifying some unpredictable phenomena into a number or string and feeding it into an algorithm which yields numbers randomly based on that seed. Easy places to find seeds include the user's mouse movements or the machine's uptime. These are only pseudorandom, however, as if given the same seed twice, the PRNG would generate the same 'random' output. This is great for games like Minecraft, but not so great for cybersecurity encryption key generation. By using a hardware random number generator (HRNG), random numbers that are not susceptible to the flaws found in PRNGs can be attained at a high rate.
Feedback Communication Over the BSC with Sparse Feedback times and Causal Encoding
Venue: arXiv cs.IT 2024
Year: 2024

Posterior matching uses variable-length encoding of the message controlled by noiseless feedback of the received symbols to achieve high rates for short average blocklengths. Traditionally, the feedback of a received symbol occurs before the next symbol is transmitted. The transmitter optimizes the next symbol transmission with full knowledge of every past received symbol. To move posterior matching closer to practical communication, this paper seeks to constrain how often feedback can be sent back to the transmitter. We focus on reducing the frequency of the feedback while still maintaining the high rates that posterior matching achieves with feedback after every symbol. As it turns out, the frequency of the feedback can be reduced significantly with no noticeable reduction in rate.
LatticeML: A data-driven application for predicting the effective Young Modulus of high temperature graph based architected materials
Venue: arXiv cs.LG 2024
Year: 2024

Architected materials with their unique topology and geometry offer the potential to modify physical and mechanical properties. Machine learning can accelerate the design and optimization of these materials by identifying optimal designs and forecasting performance. This work presents LatticeML, a data-driven application for predicting the effective Young's Modulus of high-temperature graph-based architected materials. The study considers eleven graph-based lattice structures with two high-temperature alloys, Ti-6Al-4V and Inconel 625. Finite element simulations were used to compute the effective Young's Modulus of the 2x2x2 unit cell configurations. A machine learning framework was developed to predict Young's Modulus, involving data collection, preprocessing, implementation of regression models, and deployment of the best-performing model. Five supervised learning algorithms were evaluated, with the XGBoost Regressor achieving the highest accuracy (MSE = 2.7993, MAE = 1.1521, R-squared = 0.9875). The application uses the Streamlit framework to create an interactive web interface, allowing users to input material and geometric parameters and obtain predicted Young's Modulus values.
TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals
Venue: arXiv cs.CV 2024
Year: 2024

Engagement analysis finds various applications in healthcare, education, advertisement, services. Deep Neural Networks, used for analysis, possess complex architecture and need large amounts of input data, computational power, inference time. These constraints challenge embedding systems into devices for real-time use. To address these limitations, we present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture. To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer. In parallel, to efficiently extract rich patterns from the temporal-frequency domain and boost processing speed, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form. Evaluated on the EngageNet dataset, the proposed method outperforms existing baselines, utilizing only two behavioral features (head pose rotations) compared to the 98 used in baseline models. Furthermore, comparative analysis shows TCCT-Net's architecture offers an order-of-magnitude improvement in inference speed compared to state-of-the-art image-based Recurrent Neural Network (RNN) methods. The code will be released at https://github.com/vedernikovphoto/TCCT_Net.
FreqMamba: Viewing Mamba from a Frequency Perspective for Image Deraining
Venue: arXiv cs.CV 2024
Year: 2024

Images corrupted by rain streaks often lose vital frequency information for perception, and image deraining aims to solve this issue which relies on global and local degradation modeling. Recent studies have witnessed the effectiveness and efficiency of Mamba for perceiving global and local information based on its exploiting local correlation among patches, however, rarely attempts have been explored to extend it with frequency analysis for image deraining, limiting its ability to perceive global degradation that is relevant to frequency modeling (e.g. Fourier transform). In this paper, we propose FreqMamba, an effective and efficient paradigm that leverages the complementary between Mamba and frequency analysis for image deraining. The core of our method lies in extending Mamba with frequency analysis from two perspectives: extending it with frequency-band for exploiting frequency correlation, and connecting it with Fourier transform for global degradation modeling. Specifically, FreqMamba introduces complementary triple interaction structures including spatial Mamba, frequency band Mamba, and Fourier global modeling. Frequency band Mamba decomposes the image into sub-bands of different frequencies to allow 2D scanning from the frequency dimension. Furthermore, leveraging Mamba's unique data-dependent properties, we use rainy images at different scales to provide degradation priors to the network, thereby facilitating efficient training. Extensive experiments show that our method outperforms state-of-the-art methods both visually and quantitatively.
Servers Placement Scheme Based on All-pay Auction Framework in Mobile Edge Computing
Venue: arXiv cs.GT 2024
Year: 2024

Task offloading plays a pivotal role in mobile edge computing, enabling terminal devices to enhance task execution efficiency and conserve energy. However, servers are reluctant to offer services without compensation. Currently, pricing mechanisms are commonly employed to incentivize servers to serve terminal devices, with servers earning revenue through payments from these devices. Given the rapid surge in terminal devices, determining the optimal number of servers placement for service providers (SPs) to maximize revenue is crucial. In this paper, we propose a server placement scheme based on an all-pay auction framework. Experimental simulations reveal that an optimal server-user ratio of approximately 25% maximizes SP profits.
A Legal Risk Taxonomy for Generative Artificial Intelligence
Venue: arXiv cs.CY 2024
Year: 2024

For the first time, this paper presents a taxonomy of legal risks associated with generative AI (GenAI) by breaking down complex legal concepts to provide a common understanding of potential legal challenges for developing and deploying GenAI models. The methodology is based on (1) examining the legal claims that have been filed in existing lawsuits and (2) evaluating the reasonably foreseeable legal claims that may be filed in future lawsuits. First, we identified 29 lawsuits against prominent GenAI entities and tallied the claims of each lawsuit. From there, we identified seven claims that are cited at least four times across these lawsuits as the most likely claims for future GenAI lawsuits. For each of these seven claims, we describe the elements of the claim (what the plaintiff must prove to prevail) and provide an example of how it may apply to GenAI. Next, we identified 30 other potential claims that we consider to be more speculative, because they have been included in fewer than four lawsuits or have yet to be filed. We further separated those 30 claims into 19 that are most likely to be made in relation to pre-deployment of GenAI models and 11 that are more likely to be made in connection with post-deployment of GenAI models since the legal risks will vary between entities that create versus deploy them. For each of these claims, we describe the elements of the claim and the potential remedies that plaintiffs may seek to help entities determine their legal risks in developing or deploying GenAI. Lastly, we close the paper by noting the novelty of GenAI technology and propose some applications for the paper's taxonomy in driving further research.
State Space Model for New-Generation Network Alternative to Transformers: A Survey
Venue: arXiv cs.LG 2024
Year: 2024

In the post-deep learning era, the Transformer architecture has demonstrated its powerful performance across pre-trained big models and various downstream tasks. However, the enormous computational demands of this architecture have deterred many researchers. To further reduce the complexity of attention models, numerous efforts have been made to design more efficient methods. Among them, the State Space Model (SSM), as a possible replacement for the self-attention based Transformer model, has drawn more and more attention in recent years. In this paper, we give the first comprehensive review of these works and also provide experimental comparisons and analysis to better demonstrate the features and advantages of SSM. Specifically, we first give a detailed description of principles to help the readers quickly capture the key ideas of SSM. After that, we dive into the reviews of existing SSMs and their various applications, including natural language processing, computer vision, graph, multi-modal and multi-media, point cloud/event stream, time series data, and other domains. In addition, we give statistical comparisons and analysis of these models and hope it helps the readers to understand the effectiveness of different structures on various tasks. Then, we propose possible research points in this direction to better promote the development of the theoretical model and application of SSM. More related works will be continuously updated on the following GitHub: https://github.com/Event-AHU/Mamba_State_Space_Model_Paper_List.
LR-FHSS-Sim: A Discrete-Event Simulator for LR-FHSS Networks
Venue: arXiv cs.NI 2024
Year: 2024

This work presents the LR-FHSS-Sim, a free and open-source discrete-event simulator for LR-FHSS networks. We highlight the importance of network modeling for IoT coverage, especially when it is needed to capture dynamic network behaviors. Written in Python, we present the LR-FHSS-Sim main structure, procedures, and extensions. We discuss the importance of a modular code, which facilitates the creation of algorithmic strategies and signal-processing techniques for LR-FHSS networks. Moreover, we showcase how to achieve results when considering different packet generation traffic patterns and with a previously published extension. Finally, we discuss our thoughts on future implementations and what can be achieved with them.
GNNavigator: Towards Adaptive Training of Graph Neural Networks via Automatic Guideline Exploration
Venue: arXiv cs.LG 2024
Year: 2024

Graph Neural Networks (GNNs) succeed significantly in many applications recently. However, balancing GNNs training runtime cost, memory consumption, and attainable accuracy for various applications is non-trivial. Previous training methodologies suffer from inferior adaptability and lack a unified training optimization solution. To address the problem, this work proposes GNNavigator, an adaptive GNN training configuration optimization framework. GNNavigator meets diverse GNN application requirements due to our unified software-hardware co-abstraction, proposed GNNs training performance model, and practical design space exploration solution. Experimental results show that GNNavigator can achieve up to 3.1x speedup and 44.9% peak memory reduction with comparable accuracy to state-of-the-art approaches.
Transformers, Contextualism, and Polysemy
Venue: arXiv cs.CL 2024
Year: 2024

The transformer architecture, introduced by Vaswani et al. (2017), is at the heart of the remarkable recent progress in the development of language models, including widely-used chatbots such as Chat-GPT and Claude. In this paper, I argue that we can extract from the way the transformer architecture works a theory of the relationship between context and meaning. I call this the transformer theory, and I argue that it is novel with regard to two related philosophical debates: the contextualism debate regarding the extent of context-sensitivity across natural language, and the polysemy debate regarding how polysemy should be captured within an account of word meaning.
GeoSACS: Geometric Shared Autonomy via Canal Surfaces
Venue: arXiv cs.RO 2024
Year: 2024

We introduce GeoSACS, a geometric framework for shared autonomy (SA). In variable environments, SA methods can be used to combine robotic capabilities with real-time human input in a way that offloads the physical task from the human. To remain intuitive, it can be helpful to simplify requirements for human input (i.e., reduce the dimensionality), which create challenges for to map low-dimensional human inputs to the higher dimensional control space of robots without requiring large amounts of data. We built GeoSACS on canal surfaces, a geometric framework that represents potential robot trajectories as a canal from as few as two demonstrations. GeoSACS maps user corrections on the cross-sections of this canal to provide an efficient SA framework. We extend canal surfaces to consider orientation and update the control frames to support intuitive mapping from user input to robot motions. Finally, we demonstrate GeoSACS in two preliminary studies, including a complex manipulation task where a robot loads laundry into a washer.
Enhancing Code Vulnerability Detection via Vulnerability-Preserving Data Augmentation
Venue: arXiv cs.CR 2024
Year: 2024

Source code vulnerability detection aims to identify inherent vulnerabilities to safeguard software systems from potential attacks. Many prior studies overlook diverse vulnerability characteristics, simplifying the problem into a binary (0-1) classification task for example determining whether it is vulnerable or not. This poses a challenge for a single deep learning-based model to effectively learn the wide array of vulnerability characteristics. Furthermore, due to the challenges associated with collecting large-scale vulnerability data, these detectors often overfit limited training datasets, resulting in lower model generalization performance. To address the aforementioned challenges, in this work, we introduce a fine-grained vulnerability detector namely FGVulDet. Unlike previous approaches, FGVulDet employs multiple classifiers to discern characteristics of various vulnerability types and combines their outputs to identify the specific type of vulnerability. Each classifier is designed to learn type-specific vulnerability semantics. Additionally, to address the scarcity of data for some vulnerability types and enhance data diversity for learning better vulnerability semantics, we propose a novel vulnerability-preserving data augmentation technique to augment the number of vulnerabilities. Taking inspiration from recent advancements in graph neural networks for learning program semantics, we incorporate a Gated Graph Neural Network (GGNN) and extend it to an edge-aware GGNN to capture edge-type information. FGVulDet is trained on a large-scale dataset from GitHub, encompassing five different types of vulnerabilities. Extensive experiments compared with static-analysis-based approaches and learning-based approaches have demonstrated the effectiveness of FGVulDet.
Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppression
Venue: arXiv cs.LG 2024
Year: 2024

Deep Neural Networks are prone to learning and relying on spurious correlations in the training data, which, for high-risk applications, can have fatal consequences. Various approaches to suppress model reliance on harmful features have been proposed that can be applied post-hoc without additional training. Whereas those methods can be applied with efficiency, they also tend to harm model performance by globally shifting the distribution of latent features. To mitigate unintended overcorrection of model behavior, we propose a reactive approach conditioned on model-derived knowledge and eXplainable Artificial Intelligence (XAI) insights. While the reactive approach can be applied to many post-hoc methods, we demonstrate the incorporation of reactivity in particular for P-ClArC (Projective Class Artifact Compensation), introducing a new method called R-ClArC (Reactive Class Artifact Compensation). Through rigorous experiments in controlled settings (FunnyBirds) and with a real-world dataset (ISIC2019), we show that introducing reactivity can minimize the detrimental effect of the applied correction while simultaneously ensuring low reliance on spurious features.
LoRA Dropout as a Sparsity Regularizer for Overfitting Control
Venue: arXiv cs.LG 2024
Year: 2024

Parameter-efficient fine-tuning methods, represented by LoRA, play an essential role in adapting large-scale pre-trained models to downstream tasks. However, fine-tuning LoRA-series models also faces the risk of overfitting on the training dataset, and yet there's still a lack of theoretical guidance and practical mechanism to control overfitting on LoRA-based PEFT methods. In this paper, we propose a LoRA Dropout mechanism for the LoRA-based methods by introducing random noises to the learnable low-rank matrices and increasing parameter sparsity. We then demonstrate the theoretical mechanism of our LoRA Dropout mechanism from the perspective of sparsity regularization by providing a generalization error bound under this framework. Theoretical results show that appropriate sparsity would help tighten the gap between empirical and generalization risks and thereby control overfitting. Furthermore, based on the LoRA Dropout framework, we introduce a test-time ensemble strategy and provide theoretical evidence demonstrating that the ensemble method can further compress the error bound, and lead to better performance during inference time. Extensive experiments on various NLP tasks provide practical validations of the effectiveness of our LoRA Dropout framework in improving model accuracy and calibration.
If there's a Trigger Warning, then where's the Trigger? Investigating Trigger Warnings at the Passage Level
Venue: arXiv cs.CL 2024
Year: 2024

Trigger warnings are labels that preface documents with sensitive content if this content could be perceived as harmful by certain groups of readers. Since warnings about a document intuitively need to be shown before reading it, authors usually assign trigger warnings at the document level. What parts of their writing prompted them to assign a warning, however, remains unclear. We investigate for the first time the feasibility of identifying the triggering passages of a document, both manually and computationally. We create a dataset of 4,135 English passages, each annotated with one of eight common trigger warnings. In a large-scale evaluation, we then systematically evaluate the effectiveness of fine-tuned and few-shot classifiers, and their generalizability. We find that trigger annotation belongs to the group of subjective annotation tasks in NLP, and that automatic trigger classification remains challenging but feasible.
A Review and Efficient Implementation of Scene Graph Generation Metrics
Venue: arXiv cs.CV 2024
Year: 2024

Scene graph generation has emerged as a prominent research field in computer vision, witnessing significant advancements in the recent years. However, despite these strides, precise and thorough definitions for the metrics used to evaluate scene graph generation models are lacking. In this paper, we address this gap in the literature by providing a review and precise definition of commonly used metrics in scene graph generation. Our comprehensive examination clarifies the underlying principles of these metrics and can serve as a reference or introduction to scene graph metrics. Furthermore, to facilitate the usage of these metrics, we introduce a standalone Python package called SGBench that efficiently implements all defined metrics, ensuring their accessibility to the research community. Additionally, we present a scene graph benchmarking web service, that enables researchers to compare scene graph generation methods and increase visibility of new methods in a central place. All of our code can be found at https://lorjul.github.io/sgbench/.
Privacy-Preserving Intrusion Detection using Convolutional Neural Networks
Venue: arXiv cs.CR 2024
Year: 2024

Privacy-preserving analytics is designed to protect valuable assets. A common service provision involves the input data from the client and the model on the analyst's side. The importance of the privacy preservation is fuelled by legal obligations and intellectual property concerns. We explore the use case of a model owner providing an analytic service on customer's private data. No information about the data shall be revealed to the analyst and no information about the model shall be leaked to the customer. Current methods involve costs: accuracy deterioration and computational complexity. The complexity, in turn, results in a longer processing time, increased requirement on computing resources, and involves data communication between the client and the server. In order to deploy such service architecture, we need to evaluate the optimal setting that fits the constraints. And that is what this paper addresses. In this work, we enhance an attack detection system based on Convolutional Neural Networks with privacy-preserving technology based on PriMIA framework that is initially designed for medical data.