OpenRaiser/Intern-Atlas huggingface dataset:
Object Detection based on the Collection of Geometric Evidence
Venue: arXiv cs.CV 2023
Year: 2023

Artificial objects usually have very stable shape features, which are stable, persistent properties in geometry. They can provide evidence for object recognition. Shape features are more stable and more distinguishing than appearance features, color features, grayscale features, or gradient features. The difficulty with object recognition based on shape features is that objects may differ in color, lighting, size, position, pose, and background interference, and it is not currently possible to predict all possible conditions. The variety of objects and conditions renders object recognition based on geometric features very challenging. This paper provides a method based on shape templates, which involves the selection, collection, and combination discrimination of geometric evidence of the edge segments of images, to find out the target object accurately from background, and it is able to identify the semantic attributes of each line segment of the target object. In essence, the method involves solving a global optimal combinatorial optimization problem. Although the complexity of the global optimal combinatorial optimization problem seems to be very high, there is no need to define the complex feature vector and no need for any expensive training process. It has very good generalization ability and environmental adaptability, and more solid basis for cognitive psychology than other methods. The process of collecting geometric evidence, which is simple and universal, shows considerable prospects for practical use. The experimental results prove that the method has great advantages in response to changes in the environment, invariant recognition, pinpointing the geometry of objects, search efficiency, and efficient calculation. This attempt contributes to understanding of some types of universal processing during the process of object recognition.
A Gated Cross-domain Collaborative Network for Underwater Object Detection
Venue: arXiv cs.CV 2023
Year: 2023

Underwater object detection (UOD) plays a significant role in aquaculture and marine environmental protection. Considering the challenges posed by low contrast and low-light conditions in underwater environments, several underwater image enhancement (UIE) methods have been proposed to improve the quality of underwater images. However, only using the enhanced images does not improve the performance of UOD, since it may unavoidably remove or alter critical patterns and details of underwater objects. In contrast, we believe that exploring the complementary information from the two domains is beneficial for UOD. The raw image preserves the natural characteristics of the scene and texture information of the objects, while the enhanced image improves the visibility of underwater objects. Based on this perspective, we propose a Gated Cross-domain Collaborative Network (GCC-Net) to address the challenges of poor visibility and low contrast in underwater environments, which comprises three dedicated components. Firstly, a real-time UIE method is employed to generate enhanced images, which can improve the visibility of objects in low-contrast areas. Secondly, a cross-domain feature interaction module is introduced to facilitate the interaction and mine complementary information between raw and enhanced image features. Thirdly, to prevent the contamination of unreliable generated results, a gated feature fusion module is proposed to adaptively control the fusion ratio of cross-domain information. Our method presents a new UOD paradigm from the perspective of cross-domain information interaction and fusion. Experimental results demonstrate that the proposed GCC-Net achieves state-of-the-art performance on four underwater datasets.
Matching-based Hybrid Service Trading for Task Assignment over Dynamic Mobile Crowdsensing Networks
Venue: arXiv cs.DC 2023
Year: 2023

By opportunistically engaging mobile users (workers), mobile crowdsensing (MCS) networks have emerged as important approach to facilitate sharing of sensed/gathered data of heterogeneous mobile devices. To assign tasks among workers and ensure low overheads, a series of stable matching mechanisms is introduced in this paper, which are integrated into a novel hybrid service trading paradigm consisting of futures trading mode and spot trading mode to ensure seamless MCS service provisioning. In the futures trading mode, we determine a set of long-term workers for each task through an overbooking-enabled in-advance many-to-many matching (OIA3M) mechanism, while characterizing the associated risks under statistical analysis. In the spot trading mode, we investigate the impact of fluctuations in long-term workers' resources on the violation of service quality requirements of tasks, and formalize a spot trading mode for tasks with violated service quality requirements under practical budget constraints, where the task-worker mapping is carried out via onsite many-to-many matching (O3M) and onsite many-to-one matching (OMOM). We theoretically show that our proposed matching mechanisms satisfy stability, individual rationality, fairness and computational efficiency. Comprehensive evaluations also verify the satisfaction of these properties under practical network settings, while revealing commendable performance on running time, participators' interactions, and service quality.
FastBCSD: Fast and Efficient Neural Network for Binary Code Similarity Detection
Venue: arXiv cs.CR 2023
Year: 2023

Binary code similarity detection (BCSD) has various applications, including but not limited to vulnerability detection, plagiarism detection, and malware detection. Previous research efforts mainly focus on transforming binary code to assembly code strings using reverse compilation and then using pre-trained deep learning models with large parameters to obtain feature representation vector of binary code. While these models have proven to be effective in representing binary code, their large parameter size leads to considerable computational expenses during both training and inference. In this paper, we present a lightweight neural network, called FastBCSD, that employs a dynamic instruction vector encoding method and takes only assembly code as input feature to achieve comparable accuracy to the pre-training models while reducing the computational resources and time cost. On the BinaryCorp dataset, our method achieves a similar average MRR score to the state-of-the-art pre-training-based method (jTrans), while on the BinaryCorp 3M dataset, our method even outperforms the latest technology by 0.01. Notably, FastBCSD has a much smaller parameter size (13.4M) compared to jTrans (87.88M), and its latency time is 1/5 of jTrans on NVIDIA GTX 1080Ti.
Routing One Million Customers in a Handful of Minutes
Venue: arXiv cs.DM 2023
Year: 2023

In this paper, we propose a new dataset of Capacitated Vehicle Routing Problem instances which are up to two orders of magnitude larger than those in the currently used benchmarks. Despite these sizes might not have an immediate application to real-world logistic scenarios, we believe they could foster fresh new research efforts on the design of effective and efficient algorithmic components for routing problems. We provide computational results for such instances by running FILO2, an adaptation of the FILO algorithm proposed in Accorsi and Vigo (2021), designed to handle extremely large-scale CVRP instances. Solutions for such instances are obtained by using an a standard personal computer in a considerably short computing time, thus showing the effectiveness of the acceleration and pruning techniques already proposed in FILO. Finally, results of FILO2 on well-known literature instances show that the newly introduced changes improve the overall scalability of the approach with respect to the previous FILO design.
Towards Conducting Effective Locomotion Through Hardware Transformation in Head-Mounted-Device -- A Review Study
Venue: arXiv cs.HC 2023
Year: 2023

Immersiveness is the main characteristic of Virtual Reality(VR) applications. Precise integration between hardware design and software are necessary for providing a seamless virtual experience. Allowing the user to navigate the VR scene using locomotion techniques is crucial for making such experiences `immersive'. Locomotion in VR acts as a motion tracking unit for the user and simulates their movement in the virtual scene. These movements are commonly rotational, axial or translational based on the Degree-of-Freedom (DOF) of the application. To support effective locomotion, one of the primary challenges for VR practitioners is to transform their hardware from 3-DOF to 6-DOF or vice versa. We conducted a systematic review on different motion tracking methods employed in the Head-Mounted-Devices (HMD) to understand such hardware transformation. Our review discusses the fundamental aspects of the hardware-based transformation of HMDs to conduct virtual locomotion. Our observations led us to formulate a taxonomy of the tracking methods based on system design, which can eventually be used for the hardware transformation of HMDs. Our study also captures different metrics that VR practitioners use to evaluate the hardware based on the context, performance, and significance of its usage.
Impact of Network Delay and Decision Imperfections in IoT Assisted Cruise Ship Evacuation
Venue: arXiv cs.NI 2023
Year: 2023

Major challenges of assisting passengers to safely and quickly escape from ships when an emergency occurs, include complex realistic features such as human behavior uncertainty, dynamic human traversal times, and the computation and communication delays in the systems that offer advice to users during an emergency. In this paper, we present simulations that examine the influence of these key features on evacuation performance in terms of evacuation time. The approach is based on our previously proposed lookup table-based ship passenger evacuation method, i.e., ANT. The simulation results we present show that delays in the users' reception of instructions significantly impair the effectiveness of the evacuation service. In contrast, behavior uncertainty has a weaker influence on the performance of the navigation method. In addition, these effects also vary with the extent of the behavior uncertainty, the dynamics of the traversal time distributions, and the delay in receiving directions. These findings demonstrate the importance of carefully designing evacuation systems for passenger ships in a way that takes into account all realistic features of the ship's indoor evacuation environment, including the crucial role of information technology.
DoF Upper Bound on Three-User MISO Broadcast Channel with Two Antennas and Mixed CSIT
Venue: arXiv cs.IT 2023
Year: 2023

With delayed and imperfect current channel state information at the transmitter (CSIT), namely mixed CSIT, the sum degrees-of-freedom (sum-DoF) in the two-user multiple-input multiple-output (MIMO) broadcast channel and the $K$-user multiple-input single-output (MISO) broadcast channel with not-less-than-$K$ transmit antennas have been obtained. However, the case of the three-user broadcast channel with two transmit antennas and mixed CSIT is still unexplored. In this paper, we investigate the sum-DoF upper bound of three-user MISO broadcast channel with two transmit antennas and mixed CSIT. By exploiting genie-aided signaling and extremal inequalities, we derive the sum-DoF upper bound as $(1-α)3/2 + 9α/4$, which is at most $12.5\%$ larger than the expected sum-DoF, given by $(1-α)3/2 + 2α$. This indicates that the gap may mitigate by better bounding the imperfect current CSIT counterpart.
A Multilingual Translator to SQL with Database Schema Pruning to Improve Self-Attention
Venue: arXiv cs.AI 2023
Year: 2023

Long sequences of text are challenging in the context of transformers, due to quadratic memory increase in the self-attention mechanism. As this issue directly affects the translation from natural language to SQL queries (as techniques usually take as input a concatenated text with the question and the database schema), we present techniques that allow long text sequences to be handled by transformers with up to 512 input tokens. We propose a training process with database schema pruning (removal of tables and columns names that are useless for the query of interest). In addition, we used a multilingual approach with the mT5-large model fine-tuned with a data-augmented Spider dataset in four languages simultaneously: English, Portuguese, Spanish, and French. Our proposed technique used the Spider dataset and increased the exact set match accuracy results from 0.718 to 0.736 in a validation dataset (Dev). Source code, evaluations, and checkpoints are available at: \underline{https://github.com/C4AI/gap-text2sql}.
A Neural RDE approach for continuous-time non-Markovian stochastic control problems
Venue: arXiv cs.LG 2023
Year: 2023

We propose a novel framework for solving continuous-time non-Markovian stochastic control problems by means of neural rough differential equations (Neural RDEs) introduced in Morrill et al. (2021). Non-Markovianity naturally arises in control problems due to the time delay effects in the system coefficients or the driving noises, which leads to optimal control strategies depending explicitly on the historical trajectories of the system state. By modelling the control process as the solution of a Neural RDE driven by the state process, we show that the control-state joint dynamics are governed by an uncontrolled, augmented Neural RDE, allowing for fast Monte-Carlo estimation of the value function via trajectories simulation and memory-efficient backpropagation. We provide theoretical underpinnings for the proposed algorithmic framework by demonstrating that Neural RDEs serve as universal approximators for functions of random rough paths. Exhaustive numerical experiments on non-Markovian stochastic control problems are presented, which reveal that the proposed framework is time-resolution-invariant and achieves higher accuracy and better stability in irregular sampling compared to existing RNN-based approaches.
A Spectral Perspective towards Understanding and Improving Adversarial Robustness
Venue: arXiv cs.CV 2023
Year: 2023

Deep neural networks (DNNs) are incredibly vulnerable to crafted, imperceptible adversarial perturbations. While adversarial training (AT) has proven to be an effective defense approach, the AT mechanism for robustness improvement is not fully understood. This work investigates AT from a spectral perspective, adding new insights to the design of effective defenses. In particular, we show that AT induces the deep model to focus more on the low-frequency region, which retains the shape-biased representations, to gain robustness. Further, we find that the spectrum of a white-box attack is primarily distributed in regions the model focuses on, and the perturbation attacks the spectral bands where the model is vulnerable. Based on this observation, to train a model tolerant to frequency-varying perturbation, we propose a spectral alignment regularization (SAR) such that the spectral output inferred by an attacked adversarial input stays as close as possible to its natural input counterpart. Experiments demonstrate that SAR and its weight averaging (WA) extension could significantly improve the robust accuracy by 1.14% ~ 3.87% relative to the standard AT, across multiple datasets (CIFAR-10, CIFAR-100 and Tiny ImageNet), and various attacks (PGD, C&W and Autoattack), without any extra data.
Aeroacoustic Source Localization
Venue: arXiv cs.SD 2023
Year: 2023

The deconvolutional DAMAS algorithm can effectively eliminate the misconceptions in the usually-used beamforming localization algorithm, allowing for more accurate calculation of the source location as well as the intensity. When solving a linear system of equations, the DAMAS algorithm takes into account the mutual influence of different locations, reducing or even eliminating sidelobes and producing more accurate results. This work first introduces the principles of the DAMAS algorithm. Then it applies both the beamforming algorithm and the DAMAS algorithm to simulate the localization of a single-frequency source from a 1.5 MW wind turbine, a complex line source with the text "UCAS" and a line source downstream of an airfoil trailing edge. Finally, the work presents experimental localization results of the source of a 1.5 MW wind turbine using both the beamforming algorithm and the DAMAS algorithm.
Learning Broadcast Protocols
Venue: arXiv cs.FL 2023
Year: 2023

The problem of learning a computational model from examples has been receiving growing attention. For the particularly challenging problem of learning models of distributed systems, existing results are restricted to models with a fixed number of interacting processes. In this work we look for the first time (to the best of our knowledge) at the problem of learning a distributed system with an arbitrary number of processes, assuming only that there exists a cutoff, i.e., a number of processes that is sufficient to produce all observable behaviors. Specifically, we consider fine broadcast protocols, these are broadcast protocols (BPs) with a finite cutoff and no hidden states. We provide a learning algorithm that can infer a correct BP from a sample that is consistent with a fine BP, and a minimal equivalent BP if the sample is sufficiently complete. On the negative side we show that (a) characteristic sets of exponential size are unavoidable, (b) the consistency problem for fine BPs is NP hard, and (c) that fine BPs are not polynomially predictable.
A Closer Look at Geometric Temporal Dynamics for Face Anti-Spoofing
Venue: arXiv cs.CV 2023
Year: 2023

Face anti-spoofing (FAS) is indispensable for a face recognition system. Many texture-driven countermeasures were developed against presentation attacks (PAs), but the performance against unseen domains or unseen spoofing types is still unsatisfactory. Instead of exhaustively collecting all the spoofing variations and making binary decisions of live/spoof, we offer a new perspective on the FAS task to distinguish between normal and abnormal movements of live and spoof presentations. We propose Geometry-Aware Interaction Network (GAIN), which exploits dense facial landmarks with spatio-temporal graph convolutional network (ST-GCN) to establish a more interpretable and modularized FAS model. Additionally, with our cross-attention feature interaction mechanism, GAIN can be easily integrated with other existing methods to significantly boost performance. Our approach achieves state-of-the-art performance in the standard intra- and cross-dataset evaluations. Moreover, our model outperforms state-of-the-art methods by a large margin in the cross-dataset cross-type protocol on CASIA-SURF 3DMask (+10.26% higher AUC score), exhibiting strong robustness against domain shifts and unseen spoofing types.
Im2win: An Efficient Convolution Paradigm on GPU
Venue: arXiv cs.NE 2023
Year: 2023

Convolution is the most time-consuming operation in deep neural network operations, so its performance is critical to the overall performance of the neural network. The commonly used methods for convolution on GPU include the general matrix multiplication (GEMM)-based convolution and the direct convolution. GEMM-based convolution relies on the im2col algorithm, which results in a large memory footprint and reduced performance. Direct convolution does not have the large memory footprint problem, but the performance is not on par with GEMM-based approach because of the discontinuous memory access. This paper proposes a window-order-based convolution paradigm on GPU, called im2win, which not only reduces memory footprint but also offers continuous memory accesses, resulting in improved performance. Furthermore, we apply a range of optimization techniques on the convolution CUDA kernel, including shared memory, tiling, micro-kernel, double buffer, and prefetching. We compare our implementation with the direct convolution, and PyTorch's GEMM-based convolution with cuBLAS and six cuDNN-based convolution implementations, with twelve state-of-the-art DNN benchmarks. The experimental results show that our implementation 1) uses less memory footprint by 23.1% and achieves 3.5$\times$ TFLOPS compared with cuBLAS, 2) uses less memory footprint by 32.8% and achieves up to 1.8$\times$ TFLOPS compared with the best performant convolutions in cuDNN, and 3) achieves up to 155$\times$ TFLOPS compared with the direct convolution. We further perform an ablation study on the applied optimization techniques and find that the micro-kernel has the greatest positive impact on performance.
Mutual Query Network for Multi-Modal Product Image Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Product image segmentation is vital in e-commerce. Most existing methods extract the product image foreground only based on the visual modality, making it difficult to distinguish irrelevant products. As product titles contain abundant appearance information and provide complementary cues for product image segmentation, we propose a mutual query network to segment products based on both visual and linguistic modalities. First, we design a language query vision module to obtain the response of language description in image areas, thus aligning the visual and linguistic representations across modalities. Then, a vision query language module utilizes the correlation between visual and linguistic modalities to filter the product title and effectively suppress the content irrelevant to the vision in the title. To promote the research in this field, we also construct a Multi-Modal Product Segmentation dataset (MMPS), which contains 30,000 images and corresponding titles. The proposed method significantly outperforms the state-of-the-art methods on MMPS.
Silca: Singular Caching of Homomorphic Encryption for Outsourced Databases in Cloud Computing
Venue: arXiv cs.CR 2023
Year: 2023

Ensuring the confidentiality and privacy of sensitive information in cloud computing and outsourced databases is crucial. Homomorphic encryption (HE) offers a solution by enabling computations on encrypted data without decryption, allowing secure outsourcing while maintaining data confidentiality. However, HE faces performance challenges in query-intensive databases. To address this, we propose two novel optimizations, Silca and SilcaZ, tailored to outsourced databases in cloud computing. Silca utilizes a singular caching technique to reduce computational overhead, while SilcaZ leverages modular arithmetic operations to ensure the applicability of singular caching for intensive HE operations. We prove the semantic security of Silca and SilcaZ and implement them with CKKS and BGV in HElib as MySQL loadable functions. Extensive experiments with seven real-world datasets demonstrate their superior performance compared to existing HE schemes, bridging the gap between theoretical advancements and practical applications in applying HE schemes on outsourced databases in cloud computing.
A Self-supervised Contrastive Learning Method for Grasp Outcomes Prediction
Venue: arXiv cs.RO 2023
Year: 2023

In this paper, we investigate the effectiveness of contrastive learning methods for predicting grasp outcomes in an unsupervised manner. By utilizing a publicly available dataset, we demonstrate that contrastive learning methods perform well on the task of grasp outcomes prediction. Specifically, the dynamic-dictionary-based method with the momentum updating technique achieves a satisfactory accuracy of 81.83% using data from one single tactile sensor, outperforming other unsupervised methods. Our results reveal the potential of contrastive learning methods for applications in the field of robot grasping and highlight the importance of accurate grasp prediction for achieving stable grasps.