OpenRaiser/Intern-Atlas huggingface dataset:
Co-Scale Cross-Attentional Transformer for Rearrangement Target Detection
Venue: arXiv cs.RO 2024
Year: 2024

Rearranging objects (e.g. vase, door) back in their original positions is one of the most fundamental skills for domestic service robots (DSRs). In rearrangement tasks, it is crucial to detect the objects that need to be rearranged according to the goal and current states. In this study, we focus on Rearrangement Target Detection (RTD), where the model generates a change mask for objects that should be rearranged. Although many studies have been conducted in the field of Scene Change Detection (SCD), most SCD methods often fail to segment objects with complex shapes and fail to detect the change in the angle of objects that can be opened or closed. In this study, we propose a Co-Scale Cross-Attentional Transformer for RTD. We introduce the Serial Encoder which consists of a sequence of serial blocks and the Cross-Attentional Encoder which models the relationship between the goal and current states. We built a new dataset consisting of RGB images and change masks regarding the goal and current states. We validated our method on the dataset and the results demonstrated that our method outperformed baseline methods on $F_1$-score and mean IoU.
A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning
Venue: arXiv cs.DC 2024
Year: 2024

Asynchronous Federated Learning (AFL) confronts inherent challenges arising from the heterogeneity of devices (e.g., their computation capacities) and low-bandwidth environments, both potentially causing stale model updates (e.g., local gradients) for global aggregation. Traditional approaches mitigating the staleness of updates typically focus on either adjusting the local updating or gradient compression, but not both. Recognizing this gap, we introduce a novel approach that synergizes local updating with gradient compression. Our research begins by examining the interplay between local updating frequency and gradient compression rate, and their collective impact on convergence speed. The theoretical upper bound shows that the local updating frequency and gradient compression rate of each device are jointly determined by its computing power, communication capabilities and other factors. Building on this foundation, we propose an AFL framework called FedLuck that adaptively optimizes both local update frequency and gradient compression rates. Experiments on image classification and speech recognization show that FedLuck reduces communication consumption by 56% and training time by 55% on average, achieving competitive performance in heterogeneous and low-bandwidth scenarios compared to the baselines.
Theory and Explicit Design of a Path Planner for an SE(3) Robot
Venue: arXiv cs.RO 2024
Year: 2024

We consider path planning for a rigid spatial robot with 6 degrees of freedom (6 DOFs), moving amidst polyhedral obstacles. A correct, complete and practical path planner for such a robot has never been achieved, although this is widely recognized as a key challenge in robotics. This paper provides a complete "explicit" design, down to explicit geometric primitives that are easily implementable. Our design is within an algorithmic framework for path planners, called Soft Subdivision Search (SSS). The framework is based on the twin foundations of $ε$-exactness and soft predicates, which are critical for rigorous numerical implementations. The practicality of SSS has been previously demonstrated for various robots including 5-DOF spatial robots. In this paper, we solve several significant technical challenges for SE(3) robots: (1) We first ensure the correct theory by proving a general form of the Fundamental Theorem of the SSS theory. We prove this within an axiomatic framework, thus making it easy for future applications of this theory. (2) One component of $SE(3) = R^3 \times SO(3)$ is the non-Euclidean space SO(3). We design a novel topologically correct data structure for SO(3). Using the concept of subdivision charts and atlases for SO(3), we can now carry out subdivision of SO(3). (3) The geometric problem of collision detection takes place in $R^3$, via the footprint map. Unlike sampling-based approaches, we must reason with the notion of footprints of configuration boxes, which is much harder to characterize. Exploiting the theory of soft predicates, we design suitable approximate footprints which, when combined with the highly effective feature-set technique, lead to soft predicates. (4) Finally, we make the underlying geometric computation "explicit", i.e., avoiding a general solver of polynomial systems, in order to allow a direct implementation.
Impact of Network Topology on Byzantine Resilience in Decentralized Federated Learning
Venue: arXiv cs.LG 2024
Year: 2024

Federated learning (FL) enables a collaborative environment for training machine learning models without sharing training data between users. This is typically achieved by aggregating model gradients on a central server. Decentralized federated learning is a rising paradigm that enables users to collaboratively train machine learning models in a peer-to-peer manner, without the need for a central aggregation server. However, before applying decentralized FL in real-world use training environments, nodes that deviate from the FL process (Byzantine nodes) must be considered when selecting an aggregation function. Recent research has focused on Byzantine-robust aggregation for client-server or fully connected networks, but has not yet evaluated such aggregation schemes for complex topologies possible with decentralized FL. Thus, the need for empirical evidence of Byzantine robustness in differing network topologies is evident. This work investigates the effects of state-of-the-art Byzantine-robust aggregation methods in complex, large-scale network structures. We find that state-of-the-art Byzantine robust aggregation strategies are not resilient within large non-fully connected networks. As such, our findings point the field towards the development of topology-aware aggregation schemes, especially necessary within the context of large scale real-world deployment.
Identifying Intensity of the Structure and Content in Tweets and the Discriminative Power of Attributes in Context with Referential Translation Machines
Venue: arXiv cs.CL 2024
Year: 2024

We use referential translation machines (RTMs) to identify the similarity between an attribute and two words in English by casting the task as machine translation performance prediction (MTPP) between the words and the attribute word and the distance between their similarities for Task 10 with stacked RTM models. RTMs are also used to predict the intensity of the structure and content in tweets in English, Arabic, and Spanish in Task 1 where MTPP is between the tweets and the set of words for the emotion selected from WordNet affect emotion lists. Stacked RTM models obtain encouraging results in both.
TopoLedgerBERT: Topological Learning of Ledger Description Embeddings using Siamese BERT-Networks
Venue: arXiv cs.CE 2024
Year: 2024

This paper addresses a long-standing problem in the field of accounting: mapping company-specific ledger accounts to a standardized chart of accounts. We propose a novel solution, TopoLedgerBERT, a unique sentence embedding method devised specifically for ledger account mapping. This model integrates hierarchical information from the charts of accounts into the sentence embedding process, aiming to accurately capture both the semantic similarity and the hierarchical structure of the ledger accounts. In addition, we introduce a data augmentation strategy that enriches the training data and, as a result, increases the performance of our proposed model. Compared to benchmark methods, TopoLedgerBERT demonstrates superior performance in terms of accuracy and mean reciprocal rank.
Understanding Political Communication and Political Communicators on Twitch
Venue: arXiv cs.CY 2024
Year: 2024

As new technologies rapidly reshape patterns of political communication, platforms like Twitch are transforming how people consume political information. This entertainment-oriented live streaming platform allows us to observe the impact of technologies such as ``live-streaming'' and ``streaming-chat'' on political communication. Despite its entertainment focus, Twitch hosts a variety of political actors, including politicians and pundits. This study explores Twitch politics by addressing three main questions: 1) Who are the political Twitch streamers? 2) What content is covered in political streams? 3) How do audiences of political streams interact with each other? To identify political streamers, I leveraged the Twitch API and supervised machine-learning techniques, identifying 574 political streamers. I used topic modeling to analyze the content of political streams, revealing seven broad categories of political topics and a unique pattern of communication involving context-specific ``emotes.'' Additionally, I created user-reference networks to examine interaction patterns, finding that a small number of users dominate the communication network. This research contributes to our understanding of how new social media technologies influence political communication, particularly among younger audiences.
Harnessing the Power of LLMs: Automating Unit Test Generation for High-Performance Computing
Venue: arXiv cs.SE 2024
Year: 2024

Unit testing is crucial in software engineering for ensuring quality. However, it's not widely used in parallel and high-performance computing software, particularly scientific applications, due to their smaller, diverse user base and complex logic. These factors make unit testing challenging and expensive, as it requires specialized knowledge and existing automated tools are often ineffective. To address this, we propose an automated method for generating unit tests for such software, considering their unique features like complex logic and parallel processing. Recently, large language models (LLMs) have shown promise in coding and testing. We explored the capabilities of Davinci (text-davinci-002) and ChatGPT (gpt-3.5-turbo) in creating unit tests for C++ parallel programs. Our results show that LLMs can generate mostly correct and comprehensive unit tests, although they have some limitations, such as repetitive assertions and blank test cases.
Helios: An extremely low power event-based gesture recognition for always-on smart eyewear
Venue: arXiv cs.CV 2024
Year: 2024

This paper introduces Helios, the first extremely low-power, real-time, event-based hand gesture recognition system designed for all-day on smart eyewear. As augmented reality (AR) evolves, current smart glasses like the Meta Ray-Bans prioritize visual and wearable comfort at the expense of functionality. Existing human-machine interfaces (HMIs) in these devices, such as capacitive touch and voice controls, present limitations in ergonomics, privacy and power consumption. Helios addresses these challenges by leveraging natural hand interactions for a more intuitive and comfortable user experience. Our system utilizes a extremely low-power and compact 3mmx4mm/20mW event camera to perform natural hand-based gesture recognition for always-on smart eyewear. The camera's output is processed by a convolutional neural network (CNN) running on a NXP Nano UltraLite compute platform, consuming less than 350mW. Helios can recognize seven classes of gestures, including subtle microgestures like swipes and pinches, with 91% accuracy. We also demonstrate real-time performance across 20 users at a remarkably low latency of 60ms. Our user testing results align with the positive feedback we received during our recent successful demo at AWE-USA-2024.
A Higher-Order Vampire (Short Paper)
Venue: arXiv cs.LO 2024
Year: 2024

The support for higher-order reasoning in the Vampire theorem prover has recently been completely reworked. This rework consists of new theoretical ideas, a new implementation, and a dedicated strategy schedule. The theoretical ideas are still under development, so we discuss them at a high level in this paper. We also describe the implementation of the calculus in the Vampire theorem prover, the strategy schedule construction and several empirical performance statistics.
Competitive Analysis of Online Path Selection: Impacts of Path Length, Topology, and System-Level Costs
Venue: arXiv cs.DS 2024
Year: 2024

Consider a communication network to which a sequence of self-interested users come and send requests for data transmission between nodes. This work studies the question of how to guide the path selection choices made by those online-arriving users and maximize the social welfare. Competitive analysis is the main technical tool. Specifically, the impacts of path length bounds and topology on the competitive ratio of the designed algorithm are analyzed theoretically and explored experimentally. We observe intricate and interesting relationships between the empirical performance and the studied network parameters, which shed some light on how to design the network. We also investigate the influence of system-level costs on the optimal algorithm design.
CLIMB: A Benchmark of Clinical Bias in Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) are increasingly applied to clinical decision-making. However, their potential to exhibit bias poses significant risks to clinical equity. Currently, there is a lack of benchmarks that systematically evaluate such clinical bias in LLMs. While in downstream tasks, some biases of LLMs can be avoided such as by instructing the model to answer "I'm not sure...", the internal bias hidden within the model still lacks deep studies. We introduce CLIMB (shorthand for A Benchmark of Clinical Bias in Large Language Models), a pioneering comprehensive benchmark to evaluate both intrinsic (within LLMs) and extrinsic (on downstream tasks) bias in LLMs for clinical decision tasks. Notably, for intrinsic bias, we introduce a novel metric, AssocMAD, to assess the disparities of LLMs across multiple demographic groups. Additionally, we leverage counterfactual intervention to evaluate extrinsic bias in a task of clinical diagnosis prediction. Our experiments across popular and medically adapted LLMs, particularly from the Mistral and LLaMA families, unveil prevalent behaviors with both intrinsic and extrinsic bias. This work underscores the critical need to mitigate clinical bias and sets a new standard for future evaluations of LLMs' clinical bias.
DTR: A Unified Deep Tensor Representation Framework for Multimedia Data Recovery
Venue: arXiv cs.CV 2024
Year: 2024

Recently, the transform-based tensor representation has attracted increasing attention in multimedia data (e.g., images and videos) recovery problems, which consists of two indispensable components, i.e., transform and characterization. Previously, the development of transform-based tensor representation mainly focuses on the transform aspect. Although several attempts consider using shallow matrix factorization (e.g., singular value decomposition and negative matrix factorization) to characterize the frontal slices of transformed tensor (termed as latent tensor), the faithful characterization aspect is underexplored. To address this issue, we propose a unified Deep Tensor Representation (termed as DTR) framework by synergistically combining the deep latent generative module and the deep transform module. Especially, the deep latent generative module can faithfully generate the latent tensor as compared with shallow matrix factorization. The new DTR framework not only allows us to better understand the classic shallow representations, but also leads us to explore new representation. To examine the representation ability of the proposed DTR, we consider the representative multi-dimensional data recovery task and suggest an unsupervised DTR-based multi-dimensional data recovery model. Extensive experiments demonstrate that DTR achieves superior performance compared to state-of-the-art methods in both quantitative and qualitative aspects, especially for fine details recovery.
BFLN: A Blockchain-based Federated Learning Model for Non-IID Data
Venue: arXiv cs.DC 2024
Year: 2024

As the application of federated learning becomes increasingly widespread, the issue of imbalanced training data distribution has emerged as a significant challenge. Federated learning utilizes local data stored on different training clients for model training, rather than centralizing data on a server, thereby greatly enhancing the privacy and security of training data. However, the distribution of training data across different clients may be imbalanced, with different categories of data potentially residing on different clients. This presents a challenge to traditional federated learning, which assumes data distribution is independent and identically distributed (IID). This paper proposes a Blockchain-based Federated Learning Model for Non-IID Data (BFLN), which combines federated learning with blockchain technology. By introducing a new aggregation method and incentive algorithm, BFLN enhances the model performance of federated learning on non-IID data. Experiments on public datasets demonstrate that, compared to other state-of-the-art models, BFLN improves training accuracy and provides a sustainable incentive mechanism for personalized federated learning.
SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning
Venue: arXiv cs.CV 2024
Year: 2024

Unsupervised monocular depth estimation frameworks have shown promising performance in autonomous driving. However, existing solutions primarily rely on a simple convolutional neural network for ego-motion recovery, which struggles to estimate precise camera poses in dynamic, complicated real-world scenarios. These inaccurately estimated camera poses can inevitably deteriorate the photometric reconstruction and mislead the depth estimation networks with wrong supervisory signals. In this article, we introduce SCIPaD, a novel approach that incorporates spatial clues for unsupervised depth-pose joint learning. Specifically, a confidence-aware feature flow estimator is proposed to acquire 2D feature positional translations and their associated confidence levels. Meanwhile, we introduce a positional clue aggregator, which integrates pseudo 3D point clouds from DepthNet and 2D feature flows into homogeneous positional representations. Finally, a hierarchical positional embedding injector is proposed to selectively inject spatial clues into semantic features for robust camera pose decoding. Extensive experiments and analyses demonstrate the superior performance of our model compared to other state-of-the-art methods. Remarkably, SCIPaD achieves a reduction of 22.2\% in average translation error and 34.8\% in average angular error for camera pose estimation task on the KITTI Odometry dataset. Our source code is available at \url{https://mias.group/SCIPaD}.
Vulnerability-Hunter: An Adaptive Feature Perception Attention Network for Smart Contract Vulnerabilities
Venue: arXiv cs.CR 2024
Year: 2024

Smart Contract Vulnerability Detection (SCVD) is crucial to guarantee the quality of blockchain-based systems. Graph neural networks have been shown to be effective in learning semantic representations of smart contract code and are commonly adopted by existing deep learning-based SCVD. However, the current methods still have limitations in their utilization of graph sampling or subgraph pooling based on predefined rules for extracting crucial components from structure graphs of smart contract code. These predefined rule-based strategies, typically designed using static rules or heuristics, demonstrate limited adaptability to dynamically adjust extraction strategies according to the structure and content of the graph in heterogeneous topologies of smart contract code. Consequently, these strategies may not possess universal applicability to all smart contracts, potentially leading to false positives or omissions. To address these problems, we propose AFPNet, a novel vulnerability detection model equipped with a feature perception module that has dynamic weights for comprehensive scanning of the entire smart contract code and automatic extraction of crucial code snippets (the $P$ snippets with the largest weights). Subsequently, the relationship perception attention module employs an attention mechanism to learn dependencies among these code snippets and detect smart contract vulnerabilities. The efforts made by AFPNet consistently enable the capture of crucial code snippets and enhance the performance of SCVD optimization. We conduct an evaluation of AFPNet in the several large-scale datasets with vulnerability labels. The experimental results show that our AFPNet significantly outperforms the state-of-the-art approach by 6.38\%-14.02\% in term of F1-score. The results demonstrate the effectiveness of AFPNet in dynamically extracting valuable information and vulnerability detection.
On the power of data augmentation for head pose estimation
Venue: arXiv cs.CV 2024
Year: 2024

Deep learning has been impressively successful in the last decade in predicting human head poses from monocular images. However, for in-the-wild inputs the research community relies predominantly on a single training set, 300W-LP, of semisynthetic nature without many alternatives. This paper focuses on gradual extension and improvement of the data to explore the performance achievable with augmentation and synthesis strategies further. Modeling-wise a novel multitask head/loss design which includes uncertainty estimation is proposed. Overall, the thus obtained models are small, efficient, suitable for full 6 DoF pose estimation, and exhibit very competitive accuracy.
iSign: A Benchmark for Indian Sign Language Processing
Venue: arXiv cs.CL 2024
Year: 2024

Indian Sign Language has limited resources for developing machine learning and data-driven approaches for automated language processing. Though text/audio-based language processing techniques have shown colossal research interest and tremendous improvements in the last few years, Sign Languages still need to catch up due to the need for more resources. To bridge this gap, in this work, we propose iSign: a benchmark for Indian Sign Language (ISL) Processing. We make three primary contributions to this work. First, we release one of the largest ISL-English datasets with more than 118K video-sentence/phrase pairs. To the best of our knowledge, it is the largest sign language dataset available for ISL. Second, we propose multiple NLP-specific tasks (including SignVideo2Text, SignPose2Text, Text2Pose, Word Prediction, and Sign Semantics) and benchmark them with the baseline models for easier access to the research community. Third, we provide detailed insights into the proposed benchmarks with a few linguistic insights into the workings of ISL. We streamline the evaluation of Sign Language processing, addressing the gaps in the NLP research community for Sign Languages. We release the dataset, tasks, and models via the following website: https://exploration-lab.github.io/iSign/
Research on the Acoustic Emission Source Localization Methodology in Composite Materials based on Artificial Intelligence
Venue: arXiv cs.SD 2024
Year: 2024

In this study, methodology of acoustic emission source localization in composite materials based on artificial intelligence was presented. Carbon fiber reinforced plastic was selected for specimen, and acoustic emission signal were measured using piezoelectric devices. The measured signal was wavelet-transformed to obtain scalograms, which were used as training data for the artificial intelligence model. AESLNet(acoustic emission source localization network), proposed in this study, was constructed convolutional layers in parallel due to anisotropy of the composited materials. It is regression model to detect the coordinates of acoustic emission source location. Hyper-parameter of network has been optimized by Bayesian optimization. It has been confirmed that network can detect location of acoustic emission source with an average error of 3.02mm and a resolution of 20mm.