OpenRaiser/Intern-Atlas huggingface dataset:
DM-Align: Leveraging the Power of Natural Language Instructions to Make Changes to Images
Venue: arXiv cs.CV 2024
Year: 2024

Text-based semantic image editing assumes the manipulation of an image using a natural language instruction. Although recent works are capable of generating creative and qualitative images, the problem is still mostly approached as a black box sensitive to generating unexpected outputs. Therefore, we propose a novel model to enhance the text-based control of an image editor by explicitly reasoning about which parts of the image to alter or preserve. It relies on word alignments between a description of the original source image and the instruction that reflects the needed updates, and the input image. The proposed Diffusion Masking with word Alignments (DM-Align) allows the editing of an image in a transparent and explainable way. It is evaluated on a subset of the Bison dataset and a self-defined dataset dubbed Dream. When comparing to state-of-the-art baselines, quantitative and qualitative results show that DM-Align has superior performance in image editing conditioned on language instructions, well preserves the background of the image and can better cope with long text instructions.
Binary duadic codes and their related codes with a square-root-like lower bound
Venue: arXiv cs.IT 2024
Year: 2024

Binary cyclic codes have been a hot topic for many years, and significant progress has been made in the study of this types of codes. As is well known, it is hard to construct infinite families of binary cyclic codes [n, n+1/2] with good minimum distance. In this paper, by using the BCH bound on cyclic codes, one of the open problems proposed by Liu et al. about binary cyclic codes (Finite Field Appl 91:102270, 2023) is settled. Specially, we present several families of binary duadic codes with length 2^m-1 and dimension 2^(m-1), and the minimum distances have a square-root-like lower bound. As a by-product, the parameters of their dual codes and extended codes are provided, where the latter are self-dual and doubly-even.
Online,Target-Free LiDAR-Camera Extrinsic Calibration via Cross-Modal Mask Matching
Venue: arXiv cs.RO 2024
Year: 2024

LiDAR-camera extrinsic calibration (LCEC) is crucial for data fusion in intelligent vehicles. Offline, target-based approaches have long been the preferred choice in this field. However, they often demonstrate poor adaptability to real-world environments. This is largely because extrinsic parameters may change significantly due to moderate shocks or during extended operations in environments with vibrations. In contrast, online, target-free approaches provide greater adaptability yet typically lack robustness, primarily due to the challenges in cross-modal feature matching. Therefore, in this article, we unleash the full potential of large vision models (LVMs), which are emerging as a significant trend in the fields of computer vision and robotics, especially for embodied artificial intelligence, to achieve robust and accurate online, target-free LCEC across a variety of challenging scenarios. Our main contributions are threefold: we introduce a novel framework known as MIAS-LCEC, provide an open-source versatile calibration toolbox with an interactive visualization interface, and publish three real-world datasets captured from various indoor and outdoor environments. The cornerstone of our framework and toolbox is the cross-modal mask matching (C3M) algorithm, developed based on a state-of-the-art (SoTA) LVM and capable of generating sufficient and reliable matches. Extensive experiments conducted on these real-world datasets demonstrate the robustness of our approach and its superior performance compared to SoTA methods, particularly for the solid-state LiDARs with super-wide fields of view.
A Basis-preserving Algorithm for Computing the Bezout Matrix of Newton Polynomials
Venue: arXiv cs.SC 2024
Year: 2024

This paper tackles the problem of constructing Bezout matrices for Newton polynomials in a basis-preserving approach that operates directly with the given Newton basis, thus avoiding the need for transformation from Newton basis to monomial basis. This approach significantly reduces the computational cost and also mitigates numerical instability caused by basis transformation. For this purpose, we investigate the internal structure of Bezout matrices in Newton basis and design a basis-preserving algorithm that generates the Bezout matrix in the specified basis used to formulate the input polynomials. Furthermore, we show an application of the proposed algorithm on constructing confederate resultant matrices for Newton polynomials. Experimental results demonstrate that the proposed methods perform superior to the basis-transformation-based ones.
SafePaint: Anti-forensic Image Inpainting with Domain Adaptation
Venue: arXiv cs.CV 2024
Year: 2024

Existing image inpainting methods have achieved remarkable accomplishments in generating visually appealing results, often accompanied by a trend toward creating more intricate structural textures. However, while these models excel at creating more realistic image content, they often leave noticeable traces of tampering, posing a significant threat to security. In this work, we take the anti-forensic capabilities into consideration, firstly proposing an end-to-end training framework for anti-forensic image inpainting named SafePaint. Specifically, we innovatively formulated image inpainting as two major tasks: semantically plausible content completion and region-wise optimization. The former is similar to current inpainting methods that aim to restore the missing regions of corrupted images. The latter, through domain adaptation, endeavors to reconcile the discrepancies between the inpainted region and the unaltered area to achieve anti-forensic goals. Through comprehensive theoretical analysis, we validate the effectiveness of domain adaptation for anti-forensic performance. Furthermore, we meticulously crafted a region-wise separated attention (RWSA) module, which not only aligns with our objective of anti-forensics but also enhances the performance of the model. Extensive qualitative and quantitative evaluations show our approach achieves comparable results to existing image inpainting methods while offering anti-forensic capabilities not available in other methods.
Tracking Transforming Objects: A Benchmark
Venue: arXiv cs.CV 2024
Year: 2024

Tracking transforming objects holds significant importance in various fields due to the dynamic nature of many real-world scenarios. By enabling systems accurately represent transforming objects over time, tracking transforming objects facilitates advancements in areas such as autonomous systems, human-computer interaction, and security applications. Moreover, understanding the behavior of transforming objects provides valuable insights into complex interactions or processes, contributing to the development of intelligent systems capable of robust and adaptive perception in dynamic environments. However, current research in the field mainly focuses on tracking generic objects. In this study, we bridge this gap by collecting a novel dedicated Dataset for Tracking Transforming Objects, called DTTO, which contains 100 sequences, amounting to approximately 9.3K frames. We provide carefully hand-annotated bounding boxes for each frame within these sequences, making DTTO the pioneering benchmark dedicated to tracking transforming objects. We thoroughly evaluate 20 state-of-the-art trackers on the benchmark, aiming to comprehend the performance of existing methods and provide a comparison for future research on DTTO. With the release of DTTO, our goal is to facilitate further research and applications related to tracking transforming objects.
Mamba-FETrack: Frame-Event Tracking via State Space Model
Venue: arXiv cs.CV 2024
Year: 2024

RGB-Event based tracking is an emerging research topic, focusing on how to effectively integrate heterogeneous multi-modal data (synchronized exposure video frames and asynchronous pulse Event stream). Existing works typically employ Transformer based networks to handle these modalities and achieve decent accuracy through input-level or feature-level fusion on multiple datasets. However, these trackers require significant memory consumption and computational complexity due to the use of self-attention mechanism. This paper proposes a novel RGB-Event tracking framework, Mamba-FETrack, based on the State Space Model (SSM) to achieve high-performance tracking while effectively reducing computational costs and realizing more efficient tracking. Specifically, we adopt two modality-specific Mamba backbone networks to extract the features of RGB frames and Event streams. Then, we also propose to boost the interactive learning between the RGB and Event features using the Mamba network. The fused features will be fed into the tracking head for target object localization. Extensive experiments on FELT and FE108 datasets fully validated the efficiency and effectiveness of our proposed tracker. Specifically, our Mamba-based tracker achieves 43.5/55.6 on the SR/PR metric, while the ViT-S based tracker (OSTrack) obtains 40.0/50.9. The GPU memory cost of ours and ViT-S based tracker is 13.98GB and 15.44GB, which decreased about $9.5\%$. The FLOPs and parameters of ours/ViT-S based OSTrack are 59GB/1076GB and 7MB/60MB, which decreased about $94.5\%$ and $88.3\%$, respectively. We hope this work can bring some new insights to the tracking field and greatly promote the application of the Mamba architecture in tracking. The source code of this work will be released on \url{https://github.com/Event-AHU/Mamba_FETrack}.
Joint Spectrum Partitioning and Power Allocation for Energy Efficient Semi-Integrated Sensing and Communications
Venue: arXiv cs.IT 2024
Year: 2024

With spectrum resources becoming congested and the emergence of sensing-enabled wireless applications, conventional resource allocation methods need a revamp to support communications-only, sensing-only, and integrated sensing and communication (ISaC) services together. In this letter, we propose two joint spectrum partitioning (SP) and power allocation (PA) schemes to maximize the aggregate sensing and communication performance as well as corresponding energy efficiency (EE) of a semi-ISaC system that supports all three services in a unified manner. The proposed framework captures the priority of the distinct services, impact of target clutters, power budget and bandwidth constraints, and sensing and communication quality-of-service (QoS) requirements. We reveal that the former problem is jointly convex and the latter is a non-convex problem that can be solved optimally by exploiting fractional and parametric programming techniques. Numerical results verify the effectiveness of proposed schemes and extract novel insights related to the impact of the priority and QoS requirements of distinct services on the performance of semi-ISaC networks.
Exploring the Robustness of In-Context Learning with Noisy Labels
Venue: arXiv cs.CL 2024
Year: 2024

Recently, the mysterious In-Context Learning (ICL) ability exhibited by Transformer architectures, especially in large language models (LLMs), has sparked significant research interest. However, the resilience of Transformers' in-context learning capabilities in the presence of noisy samples, prevalent in both training corpora and prompt demonstrations, remains underexplored. In this paper, inspired by prior research that studies ICL ability using simple function classes, we take a closer look at this problem by investigating the robustness of Transformers against noisy labels. Specifically, we first conduct a thorough evaluation and analysis of the robustness of Transformers against noisy labels during in-context learning and show that they exhibit notable resilience against diverse types of noise in demonstration labels. Furthermore, we delve deeper into this problem by exploring whether introducing noise into the training set, akin to a form of data augmentation, enhances such robustness during inference, and find that such noise can indeed improve the robustness of ICL. Overall, our fruitful analysis and findings provide a comprehensive understanding of the resilience of Transformer models against label noises during ICL and provide valuable insights into the research on Transformers in natural language processing. Our code is available at https://github.com/InezYu0928/in-context-learning.
ROS 2 on a Chip, Achieving Brain-Like Speeds and Efficiency in Robotic Networking
Venue: arXiv cs.RO 2024
Year: 2024

The Robot Operating System (ROS) pubsub model played a pivotal role in developing sophisticated robotic applications. However, the complexities and real-time demands of modern robotics necessitate more efficient communication solutions that are deterministic and isochronous. This article introduces a groundbreaking approach: embedding ROS 2 message-passing infrastructure directly onto a specialized hardware chip, significantly enhancing speed and efficiency in robotic communications. Our FPGA prototypes of the chip design can send or receive packages in less than 2.5 microseconds, accelerating networking communications by more than 62x on average and improving energy consumption by more than 500x when compared to traditional ROS 2 software implementations on modern CPUs. Additionally, it dramatically reduces maximum latency in ROS 2 networking communication by more than 30,000x. In situations of peak latency, our design guarantees an isochronous response within 11 microseconds, a stark improvement over the potential hundreds of milliseconds reported by modern CPU systems under similar conditions.
Contrastive Learning Method for Sequential Recommendation based on Multi-Intention Disentanglement
Venue: arXiv cs.IR 2024
Year: 2024

Sequential recommendation is one of the important branches of recommender system, aiming to achieve personalized recommended items for the future through the analysis and prediction of users' ordered historical interactive behaviors. However, along with the growth of the user volume and the increasingly rich behavioral information, how to understand and disentangle the user's interactive multi-intention effectively also poses challenges to behavior prediction and sequential recommendation. In light of these challenges, we propose a Contrastive Learning sequential recommendation method based on Multi-Intention Disentanglement (MIDCL). In our work, intentions are recognized as dynamic and diverse, and user behaviors are often driven by current multi-intentions, which means that the model needs to not only mine the most relevant implicit intention for each user, but also impair the influence from irrelevant intentions. Therefore, we choose Variational Auto-Encoder (VAE) to realize the disentanglement of users' multi-intentions. We propose two types of contrastive learning paradigms for finding the most relevant user's interactive intention, and maximizing the mutual information of positive sample pairs, respectively. Experimental results show that MIDCL not only has significant superiority over most existing baseline methods, but also brings a more interpretable case to the research about intention-based prediction and recommendation.
AdaFSNet: Time Series Classification Based on Convolutional Network with a Adaptive and Effective Kernel Size Configuration
Venue: arXiv cs.LG 2024
Year: 2024

Time series classification is one of the most critical and challenging problems in data mining, existing widely in various fields and holding significant research importance. Despite extensive research and notable achievements with successful real-world applications, addressing the challenge of capturing the appropriate receptive field (RF) size from one-dimensional or multi-dimensional time series of varying lengths remains a persistent issue, which greatly impacts performance and varies considerably across different datasets. In this paper, we propose an Adaptive and Effective Full-Scope Convolutional Neural Network (AdaFSNet) to enhance the accuracy of time series classification. This network includes two Dense Blocks. Particularly, it can dynamically choose a range of kernel sizes that effectively encompass the optimal RF size for various datasets by incorporating multiple prime numbers corresponding to the time series length. We also design a TargetDrop block, which can reduce redundancy while extracting a more effective RF. To assess the effectiveness of the AdaFSNet network, comprehensive experiments were conducted using the UCR and UEA datasets, which include one-dimensional and multi-dimensional time series data, respectively. Our model surpassed baseline models in terms of classification accuracy, underscoring the AdaFSNet network's efficiency and effectiveness in handling time series classification tasks.
Machine Learning for Blockchain Data Analysis: Progress and Opportunities
Venue: arXiv cs.CR 2024
Year: 2024

Blockchain technology has rapidly emerged to mainstream attention, while its publicly accessible, heterogeneous, massive-volume, and temporal data are reminiscent of the complex dynamics encountered during the last decade of big data. Unlike any prior data source, blockchain datasets encompass multiple layers of interactions across real-world entities, e.g., human users, autonomous programs, and smart contracts. Furthermore, blockchain's integration with cryptocurrencies has introduced financial aspects of unprecedented scale and complexity such as decentralized finance, stablecoins, non-fungible tokens, and central bank digital currencies. These unique characteristics present both opportunities and challenges for machine learning on blockchain data. On one hand, we examine the state-of-the-art solutions, applications, and future directions associated with leveraging machine learning for blockchain data analysis critical for the improvement of blockchain technology such as e-crime detection and trends prediction. On the other hand, we shed light on the pivotal role of blockchain by providing vast datasets and tools that can catalyze the growth of the evolving machine learning ecosystem. This paper serves as a comprehensive resource for researchers, practitioners, and policymakers, offering a roadmap for navigating this dynamic and transformative field.
Kernel Corrector LSTM
Venue: arXiv cs.LG 2024
Year: 2024

Forecasting methods are affected by data quality issues in two ways: 1. they are hard to predict, and 2. they may affect the model negatively when it is updated with new data. The latter issue is usually addressed by pre-processing the data to remove those issues. An alternative approach has recently been proposed, Corrector LSTM (cLSTM), which is a Read \& Write Machine Learning (RW-ML) algorithm that changes the data while learning to improve its predictions. Despite promising results being reported, cLSTM is computationally expensive, as it uses a meta-learner to monitor the hidden states of the LSTM. We propose a new RW-ML algorithm, Kernel Corrector LSTM (KcLSTM), that replaces the meta-learner of cLSTM with a simpler method: Kernel Smoothing. We empirically evaluate the forecasting accuracy and the training time of the new algorithm and compare it with cLSTM and LSTM. Results indicate that it is able to decrease the training time while maintaining a competitive forecasting accuracy.
S3-SLAM: Sparse Tri-plane Encoding for Neural Implicit SLAM
Venue: arXiv cs.CV 2024
Year: 2024

With the emergence of Neural Radiance Fields (NeRF), neural implicit representations have gained widespread applications across various domains, including simultaneous localization and mapping. However, current neural implicit SLAM faces a challenging trade-off problem between performance and the number of parameters. To address this problem, we propose sparse tri-plane encoding, which efficiently achieves scene reconstruction at resolutions up to 512 using only 2~4% of the commonly used tri-plane parameters (reduced from 100MB to 2~4MB). On this basis, we design S3-SLAM to achieve rapid and high-quality tracking and mapping through sparsifying plane parameters and integrating orthogonal features of tri-plane. Furthermore, we develop hierarchical bundle adjustment to achieve globally consistent geometric structures and reconstruct high-resolution appearance. Experimental results demonstrate that our approach achieves competitive tracking and scene reconstruction with minimal parameters on three datasets. Source code will soon be available.
Retrieval-Oriented Knowledge for Click-Through Rate Prediction
Venue: arXiv cs.IR 2024
Year: 2024

Click-through rate (CTR) prediction is crucial for personalized online services. Sample-level retrieval-based models, such as RIM, have demonstrated remarkable performance. However, they face challenges including inference inefficiency and high resource consumption due to the retrieval process, which hinder their practical application in industrial settings. To address this, we propose a universal plug-and-play \underline{r}etrieval-\underline{o}riented \underline{k}nowledge (\textbf{\name}) framework that bypasses the real retrieval process. The framework features a knowledge base that preserves and imitates the retrieved \& aggregated representations using a decomposition-reconstruction paradigm. Knowledge distillation and contrastive learning optimize the knowledge base, enabling the integration of retrieval-enhanced representations with various CTR models. Experiments on three large-scale datasets demonstrate \name's exceptional compatibility and performance, with the neural knowledge base serving as an effective surrogate for the retrieval pool. \name surpasses the teacher model while maintaining superior inference efficiency and demonstrates the feasibility of distilling knowledge from non-parametric methods using a parametric approach. These results highlight \name's strong potential for real-world applications and its ability to transform retrieval-based methods into practical solutions. Our implementation code is available to support reproducibility in \url{https://github.com/HSLiu-Initial/ROK.git}.
Multiport Network Modeling for Reconfigurable Intelligent Surfaces: Numerical Validation with a Full-Wave PEEC Simulator
Venue: arXiv cs.IT 2024
Year: 2024

Reconfigurable Intelligent Surface (RIS) modeling and optimization are a crucial steps in developing the next generation of wireless communications. To this aim, the availability of accurate electromagnetic (EM) models is of paramount important for the design of RIS-assisted communication links. In this work, we validate a widely-used analytical multiport network for RISs by means of a well-established full-wave numerical method based on the Partial Elements Equivalent Circuit (PEEC) approach. Numerical results show good agreement between the two methods, thus demonstrating i) the considered multiport network model being effective and ii) the PEEC method being appropriate for EM modeling of RIS-assisted wireless links.