Reviewerly/exHarmony huggingface dataset:
Visual Servoing Scheme and Nonlinear Model Predictive Control for Micro Quadrotor under Unknown Feature Depth

The visual servo for micro quadrotor is a complex nonlinear system with strong coupling, time-varying parameters, and underactuation. Traditional controllers are not suitable for such systems, especially in the presence of time-varying, underactuated, and state-coupled nonlinearities. The inadequacy of these controllers can result in the quadrotor moving too fast during tracking, causing motion blur to the camera, leading to tracking failure. To address these issues, we propose a robust nonlinear model predictive control (NMPC) scheme for micro quadrotor system that accounts for uncertain feature depth. First, we establish the system model using the virtual camera approach. We then design a feature depth estimator and high-gain observer to estimate the feature depth and linear velocity of the quadrotor. To ensure the system meets FOV, actuator saturation, and linear velocity constraints in the presence of external disturbances, we design tight state constraints based on the Lipschitz condition. Finally, we design an experimental platform to carry out the micro quadrotor tracking ground target experiment. The results shows that the proposed NMPC-based visual servoing scheme can achieve the stable tracking of micro quadrotor to ground target.
Research on Improved Attentional Video Instance Segmentation Algorithm

Video instance segmentation extends the concept of image instance segmentation through incorporating tracking approaches to address problems faced during object detection tasks, such as dense occlusion. Traditional Transformer-based methods, such as segmentation tracking, have a number of drawbacks, including insensitivity to small objects, sluggish convergence during training, and significant complexity. To address these issues, the Deformable Relative Inter-frame Communication Transformers (DR_IFC) approach, which is based on the efficient video instance segmentation algorithm Inter-frame Communication Transformers (IFC), was proposed as an enhanced attentional video instance segmentation algorithm. Through improvements to the attentional module and position encoding module of the video instance segmentation algorithm, the proposed method improves segmentation and tracking performance while boosting training efficiency. Specifically, we develop the deformable attention module, which focuses on the main aspects highlighted in all feature image components and effectively enhances the Transformer’s storage and processing efficiency. Furthermore, to strengthen the model’s characterization capabilities, we employ relative position encoding to explicitly characterize the position relationship of any two Tokens in the Transformer input sequence. DR_IFC algorithm was evaluated using the YouTube-VIS dataset, which includes complex and diverse scenes. Experimental results demonstrate that the proposed improved method significantly enhances segmentation and tracking performance. Without adjusting any additional hyperparameters, such as learning rate and weight decay, DR_IFC achieved consistent gains in mAP metrics of up to 4.43% over the original IFC algorithm, proving the usefulness of the upgraded attention module and location encoding module.
Research on the planning method of moving robotic path planning of ant colony optimization algorithms

When the traditional ant colony algorithm is selected to solve the robot path planning problem, there are a series of problems, such as non-convergence of results and low planning efficiency. How to improve the algorithm performance and rational path planning is crucial. Combined with the idea of elite ants, an improved ant colony algorithm based on behavioral strategy is proposed. The algorithm selects the favorable grid for the next walk in real time according to the relative position of the current grid where the ant is located and the target grid, which ensures the efficiency of the ant colony algorithm in path finding. The method of establishing and designing the grid model and the principle of the traditional ant colony algorithm are introduced and the defects of the traditional ant colony algorithm are analyzed. Based on the relative positions of ants and target points, 10 behavioral strategies are proposed and incorporated into the ant colony algorithm. Several comparison tests are conducted in raster maps with different obstacle areas and different target points. By comparing the planning results with traditional ant colony algorithm and elite ant colony algorithm, it is proved that the improved ant colony algorithm based on behavioral strategies has better performance and higher planning efficiency.
DVL Modeling Using a RVM based on Artificial Bee Colony Algorithm in SINS/DVL

For underwater Strapdown Inertial Navigation System/Doppler Velocity Log(SINS/DVL) integrated navigation system, it is a crucial factor for the performance of integrated navigation system to estimate the scale factor of DVL and its installation relationship with SINS accurately. Firstly, the theoretical model of DVL influenced by scale factor error, installation error between SINS and DVL and dynamic attitude are established. Due to the effect of dynamic attitude angle, the velocity measurement model of DVL presents great complexity and strong nonlinearity. Hence, this paper proposes a novel DVL model training approach using relevance vector machine (RVM), in which all error sources are considered in this model without separated DVL velocity measurement models. The Artificial Bee Colony (ABC) algorithm is used to optimize the key parameters of RVM. Secondly, the constraint information from GNSS and depth sensor is applied to strengthen the accuracy and generalization of the model. Finally, the performance of this method is verified with experiment in the Yellow Sea. Compared with the traditional least square estimation method based on SVD (SVD-LS), the DVL measurement model shows higher accuracy, and the positioning error of SINS/DVL integrated navigation system has been reduced from 3.1‰ of the voyage to 1.5‰ of the voyage.
Causally-Aware Intraoperative Imputation for Overall Survival Time Prediction

Previous efforts in vision community are mostly made on learning good representations from visual patterns. Beyond this, this paper emphasizes the high-level ability of causal reasoning. We thus present a case study of solving the challenging task of Overall Survival (OS) time in primary liver cancers. Critically, the prediction of OS time at the early stage remains challenging, due to the unobvious image patterns of reflecting the OS. To this end, we propose a causal inference system by leveraging the intraoperative attributes and the correlation among them, as an intermediate supervision to bridge the gap between the images and the final OS. Particularly, we build a causal graph, and train the images to estimate the intraoperative attributes for final as prediction. We present a novel Causally-aware Intraoperative Imputation Model (CAWIM) that can sequentially predict each attribute using its parent nodes in the estimated causal graph. To determine the causal directions, we propose a splitting-voting mechanism, which votes for the direction for each pair of adjacent nodes among multiple predictions obtained via causal discovery from heterogeneity. The practicability and effectiveness of our method are demonstrated by the promising results on liver cancer dataset of 361 patients with long-term observations.
Text-Guided Unsupervised Latent Transformation for Multi-Attribute Image Manipulation

Great progress has been made in StyleGAN-based image editing. To associate with preset attributes, most existing approaches focus on supervised learning for semantically meaningful latent space traversal directions, and each manipulation step is typically determined for an individual attribute. To address this limitation, we propose a Text-guided Unsupervised StyleGAN Latent Transformation (TUSLT) model, which adaptively infers a single transformation step in the latent space of StyleGAN to simultaneously manipulate multiple attributes on a given input image. Specifically, we adopt a two-stage architecture for a latent mapping network to break down the transformation process into two manageable steps. Our network first learns a diverse set of semantic directions tailored to an input image, and later nonlinearly fuses the ones associated with the target attributes to infer a residual vector. The resulting tightly interlinked two-stage architecture delivers the flexibility to handle diverse attribute combinations. By leveraging the cross-modal text-image representation of CLIP, we can perform pseudo annotations based on the semantic similarity between preset attribute text descriptions and training images, and further jointly train an auxiliary attribute classifier with the latent mapping network to provide semantic guidance. We perform extensive experiments to demonstrate that the adopted strategies contribute to the superior performance of TUSLT.
Stimulus Verification is a Universal and Effective Sampler in Multi-modal Human Trajectory Prediction

To comprehensively cover the uncertainty of the future, the common practice of multi-modal human trajectory prediction is to first generate a set/distribution of candidate future trajectories and then sample required numbers of trajectories from them as final predictions. Even though a large number of previous researches develop various strong models to predict candidate trajectories, how to effectively sample the final ones has not received much attention yet. In this paper, we propose stimulus verification, serving as a universal and effective sampling process to improve the multi-modal prediction capability, where stimulus refers to the factor in the observation that may affect the future movements such as social interaction and scene context. Stimulus verification introduces a probabilistic model, denoted as stimulus verifier, to verify the coherence between a predicted future trajectory and its corresponding stimulus. By highlighting prediction samples with better stimulus-coherence, stimulus verification ensures sampled trajectories plausible from the stimulus' point of view and therefore aids in better multi-modal prediction performance. We implement stimulus verification on five representative prediction frameworks and conduct exhaustive experiments on three widely-used benchmarks. Superior results demonstrate the effectiveness of our approach.
RILS: Masked Visual Reconstruction in Language Semantic Space

Both masked image modeling (MIM) and natural language supervision have facilitated the progress of transferable visual pre-training. In this work, we seek the synergy between two paradigms and study the emerging properties when MIM meets natural language supervision. To this end, we present a novel masked visual Reconstruction In Language semantic Space (RILS) pre-training framework, in which sentence representations, encoded by the text encoder, serve as prototypes to transform the vision-only signals into patch-sentence probabilities as semantically meaningful MIM reconstruction targets. The vision models can therefore capture useful components with structured information by predicting proper semantic of masked tokens. Better visual representations could, in turn, improve the text encoder via the image-text alignment objective, which is essential for the effective MIM target transformation. Extensive experimental results demonstrate that our method not only enjoys the best of previous MIM and CLIP but also achieves further improvements on various tasks due to their mutual benefits. RILS exhibits advanced transferability on downstream classification, detection, and segmentation, especially for low-shot regimes. Code is available at https://github.com/hustvl/RILS.
Exploring Intra-class Variation Factors with Learnable Cluster Prompts for Semi-supervised Image Synthesis

Semi-supervised class-conditional image synthesis is typically performed by inferring and injecting class labels into a conditional Generative Adversarial Network (GAN). The supervision in the form of class identity may be inadequate to model classes with diverse visual appearances. In this paper, we propose a Learnable Cluster Prompt-based GAN (LCP-GAN) to capture class-wise characteristics and intra-class variation factors with a broader source of supervision. To exploit partially labeled data, we perform soft partitioning on each class, and explore the possibility of associating intra-class clusters with learnable visual concepts in the feature space of a pre-trained language-vision model, e.g., CLIP. For class-conditional image generation, we design a cluster-conditional generator by injecting a combination of intra-class cluster label embeddings, and further incorporate a real-fake classification head on top of CLIP to distinguish real instances from the synthesized ones, conditioned on the learnable cluster prompts. This significantly strengthens the generator with more semantic language supervision. LCP-GAN not only possesses superior generation capability but also matches the performance of the fully supervised version of the base models: BigGAN and StyleGAN2-ADA, on multiple standard benchmarks.
Total Variation with Differential Privacy: Tighter Composition and Asymptotic Bounds

The framework of approximate differential privacy is considered, and augmented by introducing the notion of "the total variation of a (privacy-preserving) mechanism" (denoted by η-TV). With this refinement, an exact composition result is derived, and shown to be significantly tighter than the optimal bounds for differential privacy (which do not consider the total variation). Furthermore, it is shown that (ε, δ)-DP with η-TV is closed under subsampling. Finally, the induced total variation of commonly used mechanisms are computed.
Fusing Structural and Appearance Features for 3D Layout Estimation

3D layout estimation aims to infer the overall spatial structure of the indoor scene from an input image, and it reflects the distribution of the indoor dominant planes in a 3D space. Recent researches on layout estimation mainly learn the layout clues and features based on the visual appearance from the color images. However, little attention is paid to the depth maps that can provide the 3D structural information of the dominant planes and the relationships to the indoor objects, e.g., a bed is usually placed against a wall. Inspired by this, we first adopt a monocular depth estimation module and learn the structural feature from the estimated depth map. Then we fuse the structural feature and the appearance feature from the color image to learn the surface parameters for 3D layout inference. During training, we propose to add virtual planar obstacle in the estimated depth map, which can promote the network to put more attention on global information and also augment the training samples. Experimental results show that our method achieves state-of-the-art performance, and it proves that the depth map can provide favorable and reliable geometric clues for layout estimation.
Weakly-Supervised Deep Image Hashing based on Cross-Modal Transformer

Weakly-supervised image hashing emerges recently because web images associated with contextual text or tags are abundant. Text information weakly-related to images can be utilized to guide the learning of a deep hashing network. In this paper, we propose Weakly-supervised deep Hashing based on Cross-Modal Transformer (WHCMT). First, cross-scale attention between image patches is discovered to form more effective visual representations. A baseline transformer is also adopted to find self-attention of tags and form tag representations. Second, the cross-modal attention between images and tags is discovered by the proposed cross-modal transformer. Effective hash codes are then generated by embedding layers. WHCMT is tested on semantic image retrieval, and we show new state-of-the-art results can be obtained for the MIRFLICKR-25K dataset and NUS-WIDE dataset.
A Novel Construction Technique for Some Classes of Quasi-Cyclic Codes

Let R be an arbitrary finite commutative chain ring, γ be a fixed generator of the maximal ideal of R, and F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q</inf> = R/〈γ〉. In this paper, using the Generalized Discrete Fourier Transform, we derive the generator matrix of repeated-root quasi-cyclic codes of length n = ℓm, where q = 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sup> , m = 2m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">'</sup> , (m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">'</sup> , 2) = 1 and $or{d_{m'}}\left( {{2^t}} \right) = m' - 1$, over R. Then, we specialize this result to derive the generator matrix for simple-root quasi-cyclic codes of length n = ℓm where ord <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</inf> (q) = m − 1. Our work presented in this paper can be used to find all classes of quasi-cyclic codes whose co-index satisfies these conditions. Finally, this method is used to list some binary quasi-cyclic codes with index 2 and find optimal and self-dual codes among them.
Efficient Verification of Neural Networks Against LVM-Based Specifications

The deployment of perception systems based on neural networks in safety critical applications requires assurance on their robustness. Deterministic guarantees on network robustness require formal verification. Standard approaches for verifying robustness analyse invariance to analytically defined transformations, but not the diverse and ubiquitous changes involving object pose, scene viewpoint, occlusions, etc. To this end, we present an efficient approach for verifying specifications definable using Latent Variable Models that capture such diverse changes. The approach involves adding an invertible encoding head to the network to be verified, enabling the verification of latent space sets with minimal reconstruction overhead. We report verification experiments for three classes of proposed latent space specifications, each capturing different types of realistic input variations. Differently from previous work in this area, the proposed approach is relatively independent of input dimensionality and scales to a broad class of deep networks and real-world datasets by mitigating the inefficiency and decoder expressivity dependence in the present state-of-the-art.
A Study on Digital Rights Management with Blockchain Technology

The rapid and widespread distribution of digital resources has led to frequent piracy and infringement of digital resources, which has led to increasingly acute copyright disputes. Blockchain technology, with its decentralised, tamper-proof and anti-counterfeiting features, offers a new solution for digital copyright protection. Although blockchain technology has many advantages, there are problems such as large-scale node and system balance, centralised storage of digital works and system performance bottlenecks due to the consensus characteristics of blockchain in the process of copyright registration and copyright transactions. To solve the above problems, the author proposes an improved consensus algorithm using Byzantine fault tolerance, combined with delegated proof-of-interest consensus, to re-divide the nodes in the network, and optimise the consensus process by atmospheric consensus nodes and ordinary nodes based on voting results.
Gaze Following Based on Attention Mechanism

We humans have the ability to follow others’ gaze and estimate their gaze targets, which is called gaze following. In social communication, we are constantly doing gaze following to gain more information. With the development of society, gaze following has gained more significant applications, such as offline retail and assistive healthcare. To automate the process of gaze following, people hope that computers can also acquire this ability. However, there are relatively few relevant studies in the field of computer vision, and many studies have failed to focus on image features that are helpful for gaze following. In this paper, we propose a gaze following method based on attention mechanism, which enables the gaze direction estimation network to focus on more effective spatial features and the saliency estimation network to focus on more effective channel features. By concatenating the two networks, we ultimately obtain the predicted heatmap of the gaze point. The experimental results show that our method can effectively improve the performance of gaze following.
TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targets

Diffusion models have achieved great success in a range of tasks, such as image synthesis and molecule design. As such successes hinge on large-scale training data collected from diverse sources, the trustworthiness of these collected data is hard to control or audit. In this work, we aim to explore the vulnerabilities of diffusion models under potential training data manipulations and try to answer: How hard is it to perform Trojan attacks on well-trained diffusion models? What are the adversarial targets that such Trojan attacks can achieve? To answer these questions, we propose an effective Trojan attack against diffusion models, TrojDiff, which optimizes the Trojan diffusion and generative processes during training. In particular, we design novel transitions during the Trojan diffusion process to diffuse adversarial targets into a biased Gaussian distribution and propose a new parameterization of the Trojan generative process that leads to an effective training objective for the attack. In addition, we consider three types of adversarial targets: the Trojaned diffusion models will always output instances belonging to a certain class from the in-domain distribution (In-D2D attack), out-of-domain distribution (Out-D2D-attack), and one specific instance (D2I attack). We evaluate TrojDiff on CIFAR-10 and CelebA datasets against both DDPM and DDIM diffusion models. We show that TrojDiff always achieves high attack performance under different adversarial targets using different types of triggers, while the performance in benign environments is preserved. The code is available at https://github.com/chenweixin107/TrojDiff.
Research on bearing-only target tracking of UUV under underwater strong noise disturbance

Aiming at the problem of bearing-only target tracking (BOTT) for UUVs in complex and dynamic ocean environments, the system is susceptible to interference from strong noise. This interference can lead to large noise covariance, poor tracking accuracy, and even filter divergence. An algorithm based on robust adaptive cubature Kalman filter (RACKF) is proposed. The algorithm consists of a noise statistics estimator (NSE) and a cubature Kalman filter (CKF). To ensure the robustness of the NSE, the paper builds a fault-tolerant NSE consisting of an unbiased NSE and a biased NSE. The simulation results show that the algorithm improves the filtering accuracy and robustness under the condition of underwater strong noise disturbance, which proves the effectiveness of the algorithm proposed in this paper.