OpenRaiser/Intern-Atlas huggingface dataset:
Siamese Representation Learning for Unsupervised Relation Extraction
Venue: arXiv cs.CL 2023
Year: 2023

Unsupervised relation extraction (URE) aims at discovering underlying relations between named entity pairs from open-domain plain text without prior information on relational distribution. Existing URE models utilizing contrastive learning, which attract positive samples and repulse negative samples to promote better separation, have got decent effect. However, fine-grained relational semantic in relationship makes spurious negative samples, damaging the inherent hierarchical structure and hindering performances. To tackle this problem, we propose Siamese Representation Learning for Unsupervised Relation Extraction -- a novel framework to simply leverage positive pairs to representation learning, possessing the capability to effectively optimize relation representation of instances and retain hierarchical information in relational feature space. Experimental results show that our model significantly advances the state-of-the-art results on two benchmark datasets and detailed analyses demonstrate the effectiveness and robustness of our proposed model on unsupervised relation extraction.
CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression
Venue: arXiv cs.CV 2023
Year: 2023

Lossy image coding standards such as JPEG and MPEG have successfully achieved high compression rates for human consumption of multimedia data. However, with the increasing prevalence of IoT devices, drones, and self-driving cars, machines rather than humans are processing a greater portion of captured visual content. Consequently, it is crucial to pursue an efficient compressed representation that caters not only to human vision but also to image processing and machine vision tasks. Drawing inspiration from the efficient coding hypothesis in biological systems and the modeling of the sensory cortex in neural science, we repurpose the compressed latent representation to prioritize semantic relevance while preserving perceptual distance. Our proposed method, Compressed Perceptual Image Patch Similarity (CPIPS), can be derived at a minimal cost from a learned neural codec and computed significantly faster than DNN-based perceptual metrics such as LPIPS and DISTS.
Joint Task Scheduling and Container Image Caching in Edge Computing
Venue: arXiv cs.DC 2023
Year: 2023

In Edge Computing (EC), containers have been increasingly used to deploy applications to provide mobile users services. Each container must run based on a container image file that exists locally. However, it has been conspicuously neglected by existing work that effective task scheduling combined with dynamic container image caching is a promising way to reduce the container image download time with the limited bandwidth resource of edge nodes. To fill in such gaps, in this paper, we propose novel joint Task Scheduling and Image Caching (TSIC) algorithms, specifically: 1) We consider the joint task scheduling and image caching problem and formulate it as a Markov Decision Process (MDP), taking the communication delay, waiting delay, and computation delay into consideration; 2) To solve the MDP problem, a TSIC algorithm based on deep reinforcement learning is proposed with the customized state and action spaces and combined with an adaptive caching update algorithm. 3) A real container system is implemented to validate our algorithms. The experiments show that our strategy outperforms the existing baseline approaches by 23\% and 35\% on average in terms of total delay and waiting delay, respectively.
YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs
Venue: arXiv cs.AR 2023
Year: 2023

We address the challenges associated with deploying neural networks on CPUs, with a particular focus on minimizing inference time while maintaining accuracy. Our novel approach is to use the dataflow (i.e., computation order) of a neural network to explore data reuse opportunities using heuristic-guided analysis and a code generation framework, which enables exploration of various Single Instruction, Multiple Data (SIMD) implementations to achieve optimized neural network execution. Our results demonstrate that the dataflow that keeps outputs in SIMD registers while also maximizing both input and weight reuse consistently yields the best performance for a wide variety of inference workloads, achieving up to 3x speedup for 8-bit neural networks, and up to 4.8x speedup for binary neural networks, respectively, over the optimized implementations of neural networks today.
Active Anomaly Detection in Confined Spaces Using Ergodic Traversal of Directed Region Graphs
Venue: arXiv cs.RO 2023
Year: 2023

We provide the first step toward developing a hierarchical control-estimation framework to actively plan robot trajectories for anomaly detection in confined spaces. The space is represented globally using a directed region graph, where a region is a landmark that needs to be visited (inspected). We devise a fast mixing Markov chain to find an ergodic route that traverses this graph so that the region visitation frequency is proportional to its anomaly detection uncertainty, while satisfying the edge directionality (region transition) constraint(s). Preliminary simulation results show fast convergence to the ergodic solution and confident estimation of the presence of anomalies in the inspected regions.
Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals
Venue: arXiv cs.CL 2023
Year: 2023

Causal explanations of the predictions of NLP systems are essential to ensure safety and establish trust. Yet, existing methods often fall short of explaining model predictions effectively or efficiently and are often model-specific. In this paper, we address model-agnostic explanations, proposing two approaches for counterfactual (CF) approximation. The first approach is CF generation, where a large language model (LLM) is prompted to change a specific text concept while keeping confounding concepts unchanged. While this approach is demonstrated to be very effective, applying LLM at inference-time is costly. We hence present a second approach based on matching, and propose a method that is guided by an LLM at training-time and learns a dedicated embedding space. This space is faithful to a given causal graph and effectively serves to identify matches that approximate CFs. After showing theoretically that approximating CFs is required in order to construct faithful explanations, we benchmark our approaches and explain several models, including LLMs with billions of parameters. Our empirical results demonstrate the excellent performance of CF generation models as model-agnostic explainers. Moreover, our matching approach, which requires far less test-time resources, also provides effective explanations, surpassing many baselines. We also find that Top-K techniques universally improve every tested method. Finally, we showcase the potential of LLMs in constructing new benchmarks for model explanation and subsequently validate our conclusions. Our work illuminates new pathways for efficient and accurate approaches to interpreting NLP systems.
Speed and Density Planning for a Speed-Constrained Robot Swarm Through a Virtual Tube
Venue: arXiv cs.RO 2023
Year: 2023

The planning and control of a robot swarm in a complex environment have attracted increasing attention. To this end, the idea of virtual tubes has been taken up in our previous work. Specifically, a virtual tube with varying widths has been planned to avoid collisions with obstacles in a complex environment. Based on the planned virtual tube for a large number of speed-constrained robots, the average forward speed and density along the virtual tube are further planned in this paper to ensure safety and improve efficiency. Compared with the existing methods, the proposed method is based on global information and can be applied to traversing narrow spaces for speed-constrained robot swarms. Numerical simulations and experiments are conducted to show that the safety and efficiency of the passing-through process are improved. A video about simulations and experiments is available on https://youtu.be/lJHdMQMqSpc.
GhostEncoder: Stealthy Backdoor Attacks with Dynamic Triggers to Pre-trained Encoders in Self-supervised Learning
Venue: arXiv cs.CV 2023
Year: 2023

Within the realm of computer vision, self-supervised learning (SSL) pertains to training pre-trained image encoders utilizing a substantial quantity of unlabeled images. Pre-trained image encoders can serve as feature extractors, facilitating the construction of downstream classifiers for various tasks. However, the use of SSL has led to an increase in security research related to various backdoor attacks. Currently, the trigger patterns used in backdoor attacks on SSL are mostly visible or static (sample-agnostic), making backdoors less covert and significantly affecting the attack performance. In this work, we propose GhostEncoder, the first dynamic invisible backdoor attack on SSL. Unlike existing backdoor attacks on SSL, which use visible or static trigger patterns, GhostEncoder utilizes image steganography techniques to encode hidden information into benign images and generate backdoor samples. We then fine-tune the pre-trained image encoder on a manipulation dataset to inject the backdoor, enabling downstream classifiers built upon the backdoored encoder to inherit the backdoor behavior for target downstream tasks. We evaluate GhostEncoder on three downstream tasks and results demonstrate that GhostEncoder provides practical stealthiness on images and deceives the victim model with a high attack success rate without compromising its utility. Furthermore, GhostEncoder withstands state-of-the-art defenses, including STRIP, STRIP-Cl, and SSL-Cleanse.
Learning How to Propagate Messages in Graph Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

This paper studies the problem of learning message propagation strategies for graph neural networks (GNNs). One of the challenges for graph neural networks is that of defining the propagation strategy. For instance, the choices of propagation steps are often specialized to a single graph and are not personalized to different nodes. To compensate for this, in this paper, we present learning to propagate, a general learning framework that not only learns the GNN parameters for prediction but more importantly, can explicitly learn the interpretable and personalized propagate strategies for different nodes and various types of graphs. We introduce the optimal propagation steps as latent variables to help find the maximum-likelihood estimation of the GNN parameters in a variational Expectation-Maximization (VEM) framework. Extensive experiments on various types of graph benchmarks demonstrate that our proposed framework can significantly achieve better performance compared with the state-of-the-art methods, and can effectively learn personalized and interpretable propagate strategies of messages in GNNs.
Comics for Everyone: Generating Accessible Text Descriptions for Comic Strips
Venue: arXiv cs.CV 2023
Year: 2023

Comic strips are a popular and expressive form of visual storytelling that can convey humor, emotion, and information. However, they are inaccessible to the BLV (Blind or Low Vision) community, who cannot perceive the images, layouts, and text of comics. Our goal in this paper is to create natural language descriptions of comic strips that are accessible to the visually impaired community. Our method consists of two steps: first, we use computer vision techniques to extract information about the panels, characters, and text of the comic images; second, we use this information as additional context to prompt a multimodal large language model (MLLM) to produce the descriptions. We test our method on a collection of comics that have been annotated by human experts and measure its performance using both quantitative and qualitative metrics. The outcomes of our experiments are encouraging and promising.
Identifying Copeland Winners in Dueling Bandits with Indifferences
Venue: arXiv cs.LG 2023
Year: 2023

We consider the task of identifying the Copeland winner(s) in a dueling bandits problem with ternary feedback. This is an underexplored but practically relevant variant of the conventional dueling bandits problem, in which, in addition to strict preference between two arms, one may observe feedback in the form of an indifference. We provide a lower bound on the sample complexity for any learning algorithm finding the Copeland winner(s) with a fixed error probability. Moreover, we propose POCOWISTA, an algorithm with a sample complexity that almost matches this lower bound, and which shows excellent empirical performance, even for the conventional dueling bandits problem. For the case where the preference probabilities satisfy a specific type of stochastic transitivity, we provide a refined version with an improved worst case sample complexity.
Distributed Deterministic Exact Minimum Weight Cycle and Multi Source Shortest Paths in Near Linear Rounds in CONGEST model
Venue: arXiv cs.DC 2023
Year: 2023

We present new deterministic algorithms for computing distributed weighted minimum weight cycle (MWC) in undirected and directed graphs and distributed weighted all nodes shortest cycle (ANSC) in directed graphs. Our algorithms for these problems run in $\tilde{O}(n)$ rounds in the CONGEST model on graphs with arbitrary non-negative edge weights, matching the lower bound up to polylogarithmic factors. Before our work, no near linear rounds deterministic algorithms were known for these problems. The previous best bound for solving these problems deterministically requires an initial computation of all pairs shortest paths (APSP) on the given graph, followed by post-processing of $O(n)$ rounds, and in total takes $\tilde{O}(n^{4/3})$ rounds, using deterministic APSP~\cite{AR-SPAA20}. The main component of our new $\tilde{O}(n)$ rounds algorithms is a deterministic technique for constructing a sequence of successive blocker sets. These blocker sets are then treated as source nodes to compute $h$-hop shortest paths, which can then be used to compute candidate shortest cycles whose hop length lies in a particular range. The shortest cycles can then be obtained by selecting the cycle with the minimum weight from all these candidate cycles. Additionally using the above blocker set sequence technique, we also obtain $\tilde{O}(n)$ rounds deterministic algorithm for the multi-source shortest paths problem (MSSP) for both directed and undirected graphs, given that the size of the source set is at most $\sqrt{n}$. This new result for MSSP can be a step towards obtaining a $o(n^{4/3})$ rounds algorithm for deterministic APSP. We also believe that our new blocker set sequence technique may have potential applications for other distributed algorithms.
Sharingan: A Transformer-based Architecture for Gaze Following
Venue: arXiv cs.CV 2023
Year: 2023

Gaze is a powerful form of non-verbal communication and social interaction that humans develop from an early age. As such, modeling this behavior is an important task that can benefit a broad set of application domains ranging from robotics to sociology. In particular, Gaze Following is defined as the prediction of the pixel-wise 2D location where a person in the image is looking. Prior efforts in this direction have focused primarily on CNN-based architectures to perform the task. In this paper, we introduce a novel transformer-based architecture for 2D gaze prediction. We experiment with 2 variants: the first one retains the same task formulation of predicting a gaze heatmap for one person at a time, while the second one casts the problem as a 2D point regression and allows us to perform multi-person gaze prediction with a single forward pass. This new architecture achieves state-of-the-art results on the GazeFollow and VideoAttentionTarget datasets. The code for this paper will be made publicly available.
ECG-SL: Electrocardiogram(ECG) Segment Learning, a deep learning method for ECG signal
Venue: arXiv cs.LG 2023
Year: 2023

Electrocardiogram (ECG) is an essential signal in monitoring human heart activities. Researchers have achieved promising results in leveraging ECGs in clinical applications with deep learning models. However, the mainstream deep learning approaches usually neglect the periodic and formative attribute of the ECG heartbeat waveform. In this work, we propose a novel ECG-Segment based Learning (ECG-SL) framework to explicitly model the periodic nature of ECG signals. More specifically, ECG signals are first split into heartbeat segments, and then structural features are extracted from each of the segments. Based on the structural features, a temporal model is designed to learn the temporal information for various clinical tasks. Further, due to the fact that massive ECG signals are available but the labeled data are very limited, we also explore self-supervised learning strategy to pre-train the models, resulting significant improvement for downstream tasks. The proposed method outperforms the baseline model and shows competitive performances compared with task-specific methods in three clinical applications: cardiac condition diagnosis, sleep apnea detection, and arrhythmia classification. Further, we find that the ECG-SL tends to focus more on each heartbeat's peak and ST range than ResNet by visualizing the saliency maps.
A Model for Calculating Cost of Applying Electronic Governance and Robotic Process Automation to a Distributed Management System
Venue: arXiv cs.CY 2023
Year: 2023

Electronic Governance (eGov) and Robotic Process Automation (RPA) are two technological advancements that have the potential to revolutionize the way organizations manage their operations. When applied to Distributed Management (DM), these technologies can further enhance organizational efficiency and effectiveness. In this brief article, we present a mathematical model for calculating the cost of accomplishing a task by applying eGov and RPA in a DM system. This model is one of the first of its kind, and is expected to spark further research on cost analysis for organizational efficiency given the unprecedented advancements in electronic and automation technologies.
Helios: An Efficient Out-of-core GNN Training System on Terabyte-scale Graphs with In-memory Performance
Venue: arXiv cs.DC 2023
Year: 2023

Training graph neural networks (GNNs) on large-scale graph data holds immense promise for numerous real-world applications but remains a great challenge. Several disk-based GNN systems have been built to train large-scale graphs in a single machine. However, they often fall short in terms of performance, especially when training on terabyte-scale graphs. This is because existing disk-based systems either overly focus on minimizing the number of SSD accesses or do not fully overlap SSD accesses with GNN training, thus resulting in substantial unnecessary overhead on the CPU side and then low GPU utilization. To this end, we propose Helios, a system that can train GNN on terabyte graphs in a single machine while achieving throughput comparable with in-memory systems. To achieve this, we first present a GPU-initiated asynchronous disk IO stack, allowing the GPU to directly access graph data on SSD. This design only requires about 30% GPU cores to reach the almost maximal disk IO throughput and wastes no GPU cores between IO submission and IO completion such that the majority of GPU cores are left for other GNN kernels. Second, we design a GPU-managed heterogeneous cache that extends the cache hierarchy to heterogeneous CPU and GPU memory and thus enhances cache lookup throughput significantly by GPU parallelism. Finally, we build a deep GNN-aware pipeline that seamlessly integrates the computation and communication phases of the entire GNN training process, maximizing the utility of GPU computation cycles. Experimental results demonstrate that Helios can match the training throughput of in-memory GNN systems, even for terabyte-scale graphs. Remarkably, Helios surpasses the state-of-the-art GPU-managed baselines by up to 6.43x and exceeds CPU-managed baselines by over 182x on all terabyte-scale graphs.
PC-NeRF: Parent-Child Neural Radiance Fields under Partial Sensor Data Loss in Autonomous Driving Environments
Venue: arXiv cs.CV 2023
Year: 2023

Reconstructing large-scale 3D scenes is essential for autonomous vehicles, especially when partial sensor data is lost. Although the recently developed neural radiance fields (NeRF) have shown compelling results in implicit representations, the large-scale 3D scene reconstruction using partially lost LiDAR point cloud data still needs to be explored. To bridge this gap, we propose a novel 3D scene reconstruction framework called parent-child neural radiance field (PC-NeRF). The framework comprises two modules, the parent NeRF and the child NeRF, to simultaneously optimize scene-level, segment-level, and point-level scene representations. Sensor data can be utilized more efficiently by leveraging the segment-level representation capabilities of child NeRFs, and an approximate volumetric representation of the scene can be quickly obtained even with limited observations. With extensive experiments, our proposed PC-NeRF is proven to achieve high-precision 3D reconstruction in large-scale scenes. Moreover, PC-NeRF can effectively tackle situations where partial sensor data is lost and has high deployment efficiency with limited training time. Our approach implementation and the pre-trained models will be available at https://github.com/biter0088/pc-nerf.
A Decentralized Cooperative Navigation Approach for Visual Homing Networks
Venue: arXiv cs.RO 2023
Year: 2023

Visual homing is a lightweight approach to visual navigation. Given the stored information of an initial 'home' location, the navigation task back to this location is achieved from any other location by comparing the stored home information to the current image and extracting a motion vector. A challenge that constrains the applicability of visual homing is that the home location must be within the robot's field of view to initiate the homing process. Thus, we propose a blockchain approach to visual navigation for a heterogeneous robot team over a wide area of visual navigation. Because it does not require map data structures, the approach is useful for robot platforms with a small computational footprint, and because it leverages current visual information, it supports a resilient and adaptive path selection. Further, we present a lightweight Proof-of-Work (PoW) mechanism for reaching consensus in the untrustworthy visual homing network.
Simulation Assessment Guidelines towards Independent Safety Assurance of Autonomous Vehicles
Venue: arXiv cs.RO 2023
Year: 2023

This Simulation Assessment Guidelines document is a public guidelines document developed by the Centre of Excellence for Testing & Research of AVs - NTU (CETRAN) in collaboration with the Land Transport Authority (LTA) of Singapore. It is primarily intended to help the developers of Autonomous Vehicles (AVs) in Singapore to prepare their software simulations and provide recommendations that can ensure their readiness for independent assessment of their virtual simulation results according to the Milestone-testing framework adopted by the assessor and the local authority in Singapore, namely, CETRAN and LTA respectively.