OpenRaiser/Intern-Atlas huggingface dataset:
An Intermediate Fusion ViT Enables Efficient Text-Image Alignment in Diffusion Models
Venue: arXiv cs.CV 2024
Year: 2024

Diffusion models have been widely used for conditional data cross-modal generation tasks such as text-to-image and text-to-video. However, state-of-the-art models still fail to align the generated visual concepts with high-level semantics in a language such as object count, spatial relationship, etc. We approach this problem from a multimodal data fusion perspective and investigate how different fusion strategies can affect vision-language alignment. We discover that compared to the widely used early fusion of conditioning text in a pretrained image feature space, a specially designed intermediate fusion can: (i) boost text-to-image alignment with improved generation quality and (ii) improve training and inference efficiency by reducing low-rank text-to-image attention calculations. We perform experiments using a text-to-image generation task on the MS-COCO dataset. We compare our intermediate fusion mechanism with the classic early fusion mechanism on two common conditioning methods on a U-shaped ViT backbone. Our intermediate fusion model achieves a higher CLIP Score and lower FID, with 20% reduced FLOPs, and 50% increased training speed compared to a strong U-ViT baseline with an early fusion.
XAV: A High-Performance Regular Expression Matching Engine for Packet Processing
Venue: arXiv cs.NI 2024
Year: 2024

Regular expression matching is the core function of various network security applications such as network intrusion detection systems. With the network bandwidth increases, it is a great challenge to implement regular expression matching for line rate packet processing. To this end, a novel scheme named XAV targeting high-performance regular expression matching is proposed in this paper. XAV first employs anchor DFA to tackle the state explosion problem of DFA. Then based on anchor DFA, two techniques including pre-filtering and regex decomposition are utilized to improve the average time complexity. Through implementing XAV with an FPGA-CPU architecture, comprehensive experiments show that a high matching throughput of up to 75 Gbps can be achieved for the large and complex Snort rule-set. Compared to state-of-the-art software schemes, XAV achieves two orders of magnitude of performance improvement. While compared to state-of-the-art FPGA-based schemes, XAV achieves more than 2.5x performance improvement with the same hardware resource consumption.
Exposing the hidden layers and interplay in the quantum software stack
Venue: arXiv cs.SE 2024
Year: 2024

Current and near-future quantum computers face resource limitations due to noise and low qubit counts. Despite this, effective quantum advantage can still be achieved due to the exponential nature of bit-to-qubit conversion. However, optimizing the software architecture of these systems is essential to utilize available resources efficiently. Unfortunately, the focus on user-friendly quantum computers has obscured critical steps in the software stack, leading to ripple effects into the stack's upper layer induced by limitations in current qubit implementations. This paper unveils the hidden interplay among layers of the quantum software stack.
PE: A Poincare Explanation Method for Fast Text Hierarchy Generation
Venue: arXiv cs.CL 2024
Year: 2024

The black-box nature of deep learning models in NLP hinders their widespread application. The research focus has shifted to Hierarchical Attribution (HA) for its ability to model feature interactions. Recent works model non-contiguous combinations with a time-costly greedy search in Eculidean spaces, neglecting underlying linguistic information in feature representations. In this work, we introduce a novel method, namely Poincare Explanation (PE), for modeling feature interactions with hyperbolic spaces in a time efficient manner. Specifically, we take building text hierarchies as finding spanning trees in hyperbolic spaces. First we project the embeddings into hyperbolic spaces to elicit inherit semantic and syntax hierarchical structures. Then we propose a simple yet effective strategy to calculate Shapley score. Finally we build the the hierarchy with proving the constructing process in the projected space could be viewed as building a minimum spanning tree and introduce a time efficient building algorithm. Experimental results demonstrate the effectiveness of our approach.
Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
Venue: arXiv cs.LG 2024
Year: 2024

Federated Learning (FL) is a distributed machine learning framework in communication network systems. However, the systems' Non-Independent and Identically Distributed (Non-IID) data negatively affect the convergence efficiency of the global model, since only a subset of these data samples are beneficial for model convergence. In pursuit of this subset, a reliable approach involves determining a measure of validity to rank the samples within the dataset. In this paper, We propose the BHerd strategy which selects a beneficial herd of local gradients to accelerate the convergence of the FL model. Specifically, we map the distribution of the local dataset to the local gradients and use the Herding strategy to obtain a permutation of the set of gradients, where the more advanced gradients in the permutation are closer to the average of the set of gradients. These top portion of the gradients will be selected and sent to the server for global aggregation. We conduct experiments on different datasets, models and scenarios by building a prototype system, and experimental results demonstrate that our BHerd strategy is effective in selecting beneficial local gradients to mitigate the effects brought by the Non-IID dataset, thus accelerating model convergence.
Technical Development of a Semi-Autonomous Robotic Partition
Venue: arXiv cs.RO 2024
Year: 2024

This technical description details the design and engineering process of a semi-autonomous robotic partition. This robotic partition prototype was subsequently employed in a longer-term evaluation in-the-wild study conducted by the authors in a real-world office setting.
A comparative analysis of embedding models for patent similarity
Venue: arXiv cs.CL 2024
Year: 2024

This paper makes two contributions to the field of text-based patent similarity. First, it compares the performance of different kinds of patent-specific pretrained embedding models, namely static word embeddings (such as word2vec and doc2vec models) and contextual word embeddings (such as transformers based models), on the task of patent similarity calculation. Second, it compares specifically the performance of Sentence Transformers (SBERT) architectures with different training phases on the patent similarity task. To assess the models' performance, we use information about patent interferences, a phenomenon in which two or more patent claims belonging to different patent applications are proven to be overlapping by patent examiners. Therefore, we use these interferences cases as a proxy for maximum similarity between two patents, treating them as ground-truth to evaluate the performance of the different embedding models. Our results point out that, first, Patent SBERT-adapt-ub, the domain adaptation of the pretrained Sentence Transformer architecture proposed in this research, outperforms the current state-of-the-art in patent similarity. Second, they show that, in some cases, large static models performances are still comparable to contextual ones when trained on extensive data; thus, we believe that the superiority in the performance of contextual embeddings may not be related to the actual architecture but rather to the way the training phase is performed.
V2X-PC: Vehicle-to-everything Collaborative Perception via Point Cluster
Venue: arXiv cs.CV 2024
Year: 2024

The objective of the collaborative vehicle-to-everything perception task is to enhance the individual vehicle's perception capability through message communication among neighboring traffic agents. Previous methods focus on achieving optimal performance within bandwidth limitations and typically adopt BEV maps as the basic collaborative message units. However, we demonstrate that collaboration with dense representations is plagued by object feature destruction during message packing, inefficient message aggregation for long-range collaboration, and implicit structure representation communication. To tackle these issues, we introduce a brand new message unit, namely point cluster, designed to represent the scene sparsely with a combination of low-level structure information and high-level semantic information. The point cluster inherently preserves object information while packing messages, with weak relevance to the collaboration range, and supports explicit structure modeling. Building upon this representation, we propose a novel framework V2X-PC for collaborative perception. This framework includes a Point Cluster Packing (PCP) module to keep object feature and manage bandwidth through the manipulation of cluster point numbers. As for effective message aggregation, we propose a Point Cluster Aggregation (PCA) module to match and merge point clusters associated with the same object. To further handle time latency and pose errors encountered in real-world scenarios, we propose parameter-free solutions that can adapt to different noisy levels without finetuning. Experiments on two widely recognized collaborative perception benchmarks showcase the superior performance of our method compared to the previous state-of-the-art approaches relying on BEV maps.
AI-Generated Video Detection via Spatio-Temporal Anomaly Learning
Venue: arXiv cs.CV 2024
Year: 2024

The advancement of generation models has led to the emergence of highly realistic artificial intelligence (AI)-generated videos. Malicious users can easily create non-existent videos to spread false information. This letter proposes an effective AI-generated video detection (AIGVDet) scheme by capturing the forensic traces with a two-branch spatio-temporal convolutional neural network (CNN). Specifically, two ResNet sub-detectors are learned separately for identifying the anomalies in spatical and optical flow domains, respectively. Results of such sub-detectors are fused to further enhance the discrimination ability. A large-scale generated video dataset (GVD) is constructed as a benchmark for model training and evaluation. Extensive experimental results verify the high generalization and robustness of our AIGVDet scheme. Code and dataset will be available at https://github.com/multimediaFor/AIGVDet.
Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operations
Venue: arXiv cs.HC 2024
Year: 2024

Advancements in technology, pilot shortages, and cost pressures are driving a trend towards single-pilot and even remote operations in aviation. Considering the extensive workload and huge risks associated with single-pilot operations, the development of a Virtual Co-Pilot (V-CoP) is expected to be a potential way to ensure aviation safety. This study proposes a V-CoP concept and explores how humans and virtual assistants can effectively collaborate. A preliminary case study is conducted to explore a critical role of V-CoP, namely automated quick procedures searching, using the multimodal large language model (LLM). The LLM-enabled V-CoP integrates the pilot instruction and real-time cockpit instrumental data to prompt applicable aviation manuals and operation procedures. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis and effective retrieval of procedure information. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis (90.5%) and effective retrieval of procedure information (86.5%). The proposed V-CoP is expected to provide a foundation for future virtual intelligent assistant development, improve the performance of single pilots, and reduce the risk of human errors in aviation.
Adaptive Frequency Bin Interval in FFT via Dense Sampling Factor $α$
Venue: arXiv cs.DS 2024
Year: 2024

The Fast Fourier Transform (FFT) is a fundamental tool for signal analysis, widely used across various fields. However, traditional FFT methods encounter challenges in adjusting the frequency bin interval, which may impede accurate spectral analysis. In this study, we propose a method for adjusting the frequency bin interval in FFT by introducing a parameter $α$. We elucidate the underlying principles of the proposed method and discuss its potential applications across various contexts. Our findings suggest that the proposed method offers a promising approach to overcome the limitations of traditional FFT methods and enhance spectral analysis accuracy.
Resonant Beam Communications: A New Design Paradigm and Challenges
Venue: arXiv cs.IT 2024
Year: 2024

Resonant beam communications (RBCom), which adopt oscillating photons between two separate retroreflectors for information transmission, exhibit potential advantages over other types of wireless optical communications (WOC). However, echo interference generated by the modulated beam reflected from the receiver affects the transmission of the desired information. To tackle this challenge, a synchronization-based point-to-point RBCom system is proposed to eliminate the echo interference, and the design for the transmitter and receiver is discussed. Subsequently, the performance of the proposed RBCom is evaluated and compared with that of visible light communications (VLC) and free space optical communications (FOC). Finally, future research directions are outlined and several implementation challenges of RBCom systems are highlighted.
Looking back and forward: A retrospective and future directions on Software Engineering for systems-of-systems
Venue: arXiv cs.SE 2024
Year: 2024

Modern systems are increasingly connected and more integrated with other existing systems, giving rise to \textit{systems-of-systems} (SoS). An SoS consists of a set of independent, heterogeneous systems that interact to provide new functionalities and accomplish global missions through emergent behavior manifested at runtime. The distinctive characteristics of SoS, when contrasted to traditional systems, pose significant research challenges within Software Engineering. These challenges motivate the need for a paradigm shift and the exploration of novel approaches for designing, developing, deploying, and evolving these systems. The \textit{International Workshop on Software Engineering for Systems-of-Systems} (SESoS) series started in 2013 to fill a gap in scientific forums addressing SoS from the Software Engineering perspective, becoming the first venue for this purpose. This article presents a study aimed at outlining the evolution and future trajectory of Software Engineering for SoS based on the examination of 57 papers spanning the 11 editions of the SESoS workshop (2013-2023). The study combined scoping review and scientometric analysis methods to categorize and analyze the research contributions concerning temporal and geographic distribution, topics of interest, research methodologies employed, application domains, and research impact. Based on such a comprehensive overview, this article discusses current and future directions in Software Engineering for SoS.
Enhancing Software Effort Estimation through Reinforcement Learning-based Project Management-Oriented Feature Selection
Venue: arXiv cs.SE 2024
Year: 2024

Purpose: The study aims to investigate the application of the data element market in software project management, focusing on improving effort estimation by addressing challenges faced by traditional methods. Design/methodology/approach: This study proposes a solution based on feature selection, utilizing the data element market and reinforcement learning-based algorithms to enhance the accuracy of software effort estimation. It explores the application of the MARLFS algorithm, customizing improvements to the algorithm and reward function. Findings: This study demonstrates that the proposed approach achieves more precise estimation compared to traditional methods, leveraging feature selection to guide project management in software development. Originality/value: This study contributes to the field by offering a novel approach that combines the data element market, machine learning, and feature selection to improve software effort estimation, addressing limitations of traditional methods and providing insights for future research in project management.
On Policy Reuse: An Expressive Language for Representing and Executing General Policies that Call Other Policies
Venue: arXiv cs.AI 2024
Year: 2024

Recently, a simple but powerful language for expressing and learning general policies and problem decompositions (sketches) has been introduced in terms of rules defined over a set of Boolean and numerical features. In this work, we consider three extensions of this language aimed at making policies and sketches more flexible and reusable: internal memory states, as in finite state controllers; indexical features, whose values are a function of the state and a number of internal registers that can be loaded with objects; and modules that wrap up policies and sketches and allow them to call each other by passing parameters. In addition, unlike general policies that select state transitions rather than ground actions, the new language allows for the selection of such actions. The expressive power of the resulting language for policies and sketches is illustrated through a number of examples.
An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
Venue: arXiv cs.CL 2024
Year: 2024

We present Expert-Token-Routing, a unified generalist framework that facilitates seamless integration of multiple expert LLMs. Our framework represents expert LLMs as special expert tokens within the vocabulary of a meta LLM. The meta LLM can route to an expert LLM like generating new tokens. Expert-Token-Routing not only supports learning the implicit expertise of expert LLMs from existing instruction dataset but also allows for dynamic extension of new expert LLMs in a plug-and-play manner. It also conceals the detailed collaboration process from the user's perspective, facilitating interaction as though it were a singular LLM. Our framework outperforms various existing multi-LLM collaboration paradigms across benchmarks that incorporate six diverse expert domains, demonstrating effectiveness and robustness in building generalist LLM system via synergizing multiple expert LLMs.
Lessons Learned from Building Edge Software System Testbeds
Venue: arXiv cs.DC 2024
Year: 2024

Edge computing requires the complex software interaction of geo-distributed, heterogeneous components. The growing research and industry interest in edge computing software systems has necessitated exploring ways of testing and evaluating edge software at scale without relying on physical infrastructure. Beyond simulation, virtual testbeds that emulate edge infrastructure can provide a cost-efficient yet realistic environment to evaluate edge software. In this experience paper, we share lessons learned from building a total of five edge software testbeds. We describe pitfalls in architecture and development as well as experiences from having students use our testbed tooling in distributed systems prototyping classes. While we remain confident that building custom testbed tooling is the right approach for edge computing researchers and practitioners alike, we hope this paper allows others to avoid common mistakes and benefit from our experience.
TAIL: A Terrain-Aware Multi-Modal SLAM Dataset for Robot Locomotion in Deformable Granular Environments
Venue: arXiv cs.RO 2024
Year: 2024

Terrain-aware perception holds the potential to improve the robustness and accuracy of autonomous robot navigation in the wilds, thereby facilitating effective off-road traversals. However, the lack of multi-modal perception across various motion patterns hinders the solutions of Simultaneous Localization And Mapping (SLAM), especially when confronting non-geometric hazards in demanding landscapes. In this paper, we first propose a Terrain-Aware multI-modaL (TAIL) dataset tailored to deformable and sandy terrains. It incorporates various types of robotic proprioception and distinct ground interactions for the unique challenges and benchmark of multi-sensor fusion SLAM. The versatile sensor suite comprises stereo frame cameras, multiple ground-pointing RGB-D cameras, a rotating 3D LiDAR, an IMU, and an RTK device. This ensemble is hardware-synchronized, well-calibrated, and self-contained. Utilizing both wheeled and quadrupedal locomotion, we efficiently collect comprehensive sequences to capture rich unstructured scenarios. It spans the spectrum of scope, terrain interactions, scene changes, ground-level properties, and dynamic robot characteristics. We benchmark several state-of-the-art SLAM methods against ground truth and provide performance validations. Corresponding challenges and limitations are also reported. All associated resources are accessible upon request at \url{https://tailrobot.github.io/}.
Movable-Antenna Position Optimization: A Graph-based Approach
Venue: arXiv cs.IT 2024
Year: 2024

Fluid antennas (FAs) and movable antennas (MAs) have emerged as promising technologies in wireless communications, which offer the flexibility to improve channel conditions by adjusting transmit/receive antenna positions within a spatial region. In this letter, we focus on an MA-enhanced multiple-input single-output (MISO) communication system, aiming to optimize the positions of multiple transmit MAs to maximize the received signal power. Unlike the prior works on continuously searching for the optimal MA positions, we propose to sample the transmit region into discrete points, such that the continuous antenna position optimization problem is transformed to a discrete sampling point selection problem based on the point-wise channel information. However, such a point selection problem is combinatory and challenging to be optimally solved. To tackle this challenge, we ingeniously recast it as an equivalent fixed-hop shortest path problem in graph theory and propose a customized algorithm to solve it optimally in polynomial time. To further reduce the complexity, a linear-time sequential update algorithm is also proposed to obtain a high-quality suboptimal solution. Numerical results demonstrate that the proposed algorithms can yield considerable performance gains over the conventional fixed-position antennas with/without antenna selection.
ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature
Venue: arXiv cs.DL 2024
Year: 2024

The use of ChatGPT and similar Large Language Model (LLM) tools in scholarly communication and academic publishing has been widely discussed since they became easily accessible to a general audience in late 2022. This study uses keywords known to be disproportionately present in LLM-generated text to provide an overall estimate for the prevalence of LLM-assisted writing in the scholarly literature. For the publishing year 2023, it is found that several of those keywords show a distinctive and disproportionate increase in their prevalence, individually and in combination. It is estimated that at least 60,000 papers (slightly over 1% of all articles) were LLM-assisted, though this number could be extended and refined by analysis of other characteristics of the papers or by identification of further indicative keywords.