Reviewerly/exHarmony huggingface dataset:
MSED: A Robust Ellipse Detector with Multi-scale Merging and Validation

In the task of ellipse detection, the edge pixels that share the similar elliptic convexity are identified and combined to produce a potential ellipse. Due to the discreteness error and various kinds of image noise inherited in real-world images (e.g., the noise caused by compression attack and low-illustration), some of these elliptic edge pixels might deviate from the positions they supposed to be. Consequently, the ellipse detectors could have inferior performance in terms of precision and recall rate. For that, the existing ellipse detection methods always apply a Gaussian smoothing to the input image before the detection procedure. Due to the scale value (i.e., the standard deviation of Gaussian kernel) needs to be determined in advance, the existing single-scale ellipse detectors still suffer from the inferior precision and recall rate issues. To simultaneously solve these issues, a multi-scale strategy is innovatively developed for ellipse detection, based on which a novel multi-scale ellipse detector (MSED) is proposed. In our MSED, the elliptic edge pixels at multiple scales are jointly used to produce ellipses with high quality. For that, the ellipses detected at multiple scales are first produced, followed by merging them with a new multi-scale ellipse merging approach. In this approach, a probabilistic model is developed, based on which the homologous ellipses at multiple scales are merged. Lastly, a multi-scale ellipse validation check is further developed to discard those merged ellipses that have low confidence. Extensive experimental results show that our MSED outperforms the current state-of-the-arts and is robust to noise.
Hierarchical Focused Feature Pyramid Network for Small Object Detection

Small object detection has been a persistently practical and challenging task in the field of computer vision. Advanced detectors often utilize a feature pyramid network (FPN) to fuse the features generated from various receptive fields, which improve the detection ability of multi-scale objects, especially for small objects. However, existing FPNs typically employ a naive addition-based fusion strategy, which neglects crucial details that may exist only at specific levels. These details are vital for accurately detecting small objects. In this paper, we propose a novel Hierarchical Focused Feature Pyramid Network (HFFPN) to enhance these details while ensuring the detection performance for objects of other scales. HFFPN consists of two key components: Hierarchical Feature Subtraction Module (HFSM) and Feature Fusion Guidance Attention (FFGA). HFSM is first designed to selectively amplify the information important to small object detection. FFGA is devised to focus on effective features by utilizing global information and mining small objects’ information from high-level features. Combining these two modules contributes greatly to the original FPN. In particular, the proposed HFFPN can be incorporated into most mainstream detectors, such as Faster RCNN, Retinanet, FCOS, etc. Extensive experiments on small object datasets demonstrate that HFFPN achieves consistent and significant improvements over the baseline algorithm while surpassing the state-of-the-art methods.
Enhancing GAN Compression by Image Probability Distribution Distillation

This paper presents a novel approach named Image Probability Distribution Distillation (IPDD) for compressing generative adversarial networks (GANs) by distilling knowledge from the global distribution of images. Unlike traditional methods that distill at the pixel level, we propose a holistic approach that captures the overall coherence of images. To achieve this, we introduce a novel teacher discriminator that engages in adversarial training with both the teacher generator and the student generator in asynchronous weighted manner, using variable weights to optimize the Nash equilibrium between them. Our framework explores the uncharted territory of mining global distribution information and federated training of the teacher discriminator, offering potential for enhancing the performance of compressed GANs. Extensive experiments on benchmark datasets demonstrate that our approach significantly reduces GAN complexity while achieving optimal performance.
GridIIS: Grid Based Interactive Image Segmentation

Interactive segmentation enables users to specify the object of interest (OOI) via various interaction strategies to obtain accurate segmentation results. An ideal interactive method should efficiently and accurately express users’ segmentation intentions. However, the existing methods can only use a single interactive mode, ignoring the differences in scale and shape between OOIs, resulting in an inflexibility labeling process. In this paper, we propose a grid-based interactive image segmentation method (GridIIS). Specifically, GridIIS overlays grids on the image, and users can specify the location and shape of the OOI by selecting the grid areas as the interactive guidance. Users can choose the appropriate grid selection method and size considering the OOI’s scale, shape, and boundary clarity to obtain guidance. We accordingly propose a novel grid sampling strategy, that considers the OOI’s scale and shapes to adaptively estimate the grid size and area. Experiments on several datasets from different domains (street views, medical images, scene texts, etc.) show that our method achieves superior performance with fewer interaction rounds and exhibits strong generalization ability in cross-domain datasets.
Boosting Object Detection in Foggy Scenes via Dark Channel Map and Union Training Strategy

Most existing object detection methods in real-world hazy scenarios fail to handle the heterogeneous haze and treat clear images and hazy images as adversarial while ignoring the latent information beneficial in clear images for detection, resulting in sub-optimal performance. To alleviate the above problems, we propose a new dark channel map-guided detection paradigm (DG-Net) in an end-to-end manner and provide an interpretable idea for object detection in hazy scenes from an entirely new perspective. Specifically, we design a unique dark channel map-guided feature fusion (DGFF) module to handle the adverse impact of the heterogeneous haze, which enables the model to focus on potential regions that may contain detection objects adaptively, assign higher weights to these regions, and thus improve the network’s ability to learn and represent the features of hazy images. To more effectively utilize the latent features of clear images, we propose a new simple but effective union training strategy (UTS) that considers the clear images as a complement to the hazy images, which enables the DGFF module to work better. In addition, we introduce Focal loss and Self-calibrated convolutions to enhance the performance of the DG-Net. Extensive experiments show that DG-Net outperforms the state-of-the-art detection methods quantitatively and qualitatively, especially in real-world hazy datasets.
Feature Refinement from Multiple Perspectives for High Performance Salient Object Detection

Recently, deep-learning based salient object detection methods have gained great progress. However, there still exists some problems such as inefficient multi-level feature fusion, unstable multi-scale context-aware feature extraction, detail loss caused by upsampling and unbalanced distribution. To efficiently fuse multi-level features, we propose an attention-guided bi-directional feature refinement module (ABFRM) including top-down and bottom-up processes, which applies different attention-based feature fusion strategies for different directional processes. To obtain stable multi-scale contextual features, we design a serial atrous fusion module (SAFM), which uses serial atrous convolutional layers with small dilation rates. To reduce detail loss caused by upsampling with a large factor, we devise an upsampling feature refinement module (UFRM), which utilizes the combination of deconvolution and bilinear interpolation. To address unbalanced distribution from both foreground and background perspectives, we propose a novel hybrid loss, which contains Intersection-over-Union (IoU) and background boundary (BGB) losses. Comprehensive experiments on five benchmark datasets demonstrate that our proposed method outperforms 13 state-of-the-art approaches under four evaluation metrics. The code is available at https://github.com/xuanli01/PRCV210 .
Generalized Zero-Shot Learning with Noisy Labeled Data

Generalized zero-shot learning (GZSL) is a challenging task that aims to classify samples with both seen and unseen categories. In practice, noisy labels can significantly degrade the performance of GZSL classifiers, which has received little attention in previous research. When noisy labeled data exists, the class-level semantic representation may fail to accurately describe some samples of the corresponding class. At the same time, noisy data can also disturb the original distributions of the corresponding classes, leading to estimation errors in modeling the sample distributions. To address these issues, we propose a novel method that aims to alleviate the influence of noisy samples in GZSL. Specifically, we propose a sample-level semantic generation method to ensure an accurate description of the corresponding sample. Furthermore, we introduce an unbalanced learning framework to address the sample distribution estimation with noisy labels to make the estimation error on each class and dimension balanced and dynamically mitigate the negative effects of the error distributions from multiple classes. Experimental results on benchmark datasets demonstrate that our approach effectively mitigates the influence of noisy samples and outperforms other advanced methods.
Modality Balancing Mechanism for RGB-Infrared Object Detection in Aerial Image

RGB-Infrared object detection in aerial images has gained significant attention due to its effectiveness in mitigating the challenges posed by illumination restrictions. Existing methods often focus heavily on enhancing the fusion of two modalities while ignoring the optimization imbalance caused by inherent differences between modalities. In this work, we observe that there is an inconsistency between two modalities during joint training, and this hampers the model’s performance. Inspired by these findings, we argue that the focus of RGB-Infrared detection should be shifted to the optimization of two modalities, and further propose a Modality Balancing Mechanism (MBM) method for training the detection model. To be specific, we initially introduce an auxiliary detection head to inspect the training process of both modalities. Subsequently, the learning rates of the two backbones are dynamically adjusted using the Scaled Gaussian Function (SGF). Furthermore, the Multi-modal Feature Hybrid Sampling Module (MHSM) is introduced to augment representation by combining complementary features extracted from both modalities. Benefiting from the design of the proposed mechanism, experimental results on DroneVehicle and LLVIP demonstrate that our approach achieves state-of-the-art performance. The code are available at ( https://github.com/ccccwb/Multimodal-Detection-and-Tracking-UAV ).
Emphasizing Boundary-Positioning and Leveraging Multi-scale Feature Fusion for Camouflaged Object Detection

Camouflaged object detection (COD) aims to identify objects that blend in with their surroundings and have numerous practical applications. However, COD is a challenging task due to the high similarity between camouflaged objects and their surroundings. To address the problem of identifying camouflaged objects, we investigated how humans observe such objects. We found that humans typically first scan the entire image to obtain an approximate location of the target object. They then observe the differences between the boundary of the target object and its surrounding environment to refine their perception of the object. This continuous refinement process helps humans eventually identify the camouflaged object. Based on this observation, we propose a novel COD method that emphasizes boundary positioning and leverages multi-scale feature fusion. Our model includes two important modules: the Enhanced Feature Module (EFM) and the Boundary and Positioning joint-guided Feature Fusion Module (BPFM). The EFM provides multi-scale information and obtains aggregated feature representations, resulting in more robust feature representations for the initial positioning of the camouflaged object. In BPFM, we mimic human observation of camouflaged objects by injecting boundary and positioning information into each level of the backbone features, working together to refine the target object in blurred regions progressively. We validated the effectiveness of our model on three benchmark datasets (COD10K, CAMO, CHAMELEON), and the results showed that our proposed method significantly outperforms existing COD models.
Boosting One-Stage Multi Object Tracking with Attention Learning

One-stage multi-object tracking methods have achieved promising results by showing their great balance between accuracy and speed. However, the internal differences and relationships between detection and re-identification (re-ID) lead to worse performance. In this work, we propose a one-stage multi-object tracking method with attention boosting, namely AeMOT, which can effectively improve the collaboration and performance in detection and re-ID. Specifically, a discriminability enhancement module is designed to enhance the discriminative feature representations for detection and tracking, and an identity preserving module is properly designed to preserve the semantic alignment of id-embedding and improve the adaptiveness of object matching with scale variation for re-ID association. Experimental results on challenging benchmarks including MOT17 and MOT20 demonstrate that our proposed method achieves leading performance and outperforms state-of-the-art trackers.
Instance-Proxy Loss for Semi-supervised Learning with Coarse Labels

Objects are often organized in a hierarchy where coarse-grained categories are comprised of subordinate fine-grained classes. Comparing with the fine-grained labels, the coarse-grained labels are much affordable to obtain. The coarse-grained labels can boost the semi-supervised learning (SSL) by offering extra regularization on the feature space of finer-grained recognition. However, coarse-grained labels are ignored by most of works in SSL. An intuitive way to utilize the coarse labels for SSL is to impose an extra coarse-grained categorization constraint, which will cause the class confusion between fine-grained categories belonging to the same coarse-grained category thus is sub-optimal for SSL. In this paper, we present an instance-proxy loss (IPL) to boost the separability of the fine-grained classes within the same coarse-grained class, as well as keep the intra-class feature space of coarse-grained classes compact. Specifically, IPL includes instance-level loss and proxy-level loss to impose constraints on both instance-to-instance and instance-to-proxy relations. Our approach outperforms the state-of-the-art methods on three benchmark datasets, showing significant improvement with small proportion of fine-grained labels, e.g., it brings 10.14% accuracy improvement on CUB-200-2011 with 15% of labeled data.
MVP-SEG: Multi-view Prompt Learning for Open-Vocabulary Semantic Segmentation

CLIP (Contrastive Language-Image Pretraining) is well developed for open-vocabulary zero-shot image-level recognition, while its applications in pixel-level tasks are less investigated, where most efforts directly adopt CLIP features without deliberative adaptations. In this work, we first demonstrate the necessity of image-pixel CLIP feature adaption, then provide Multi-View Prompt learning (MVP-SEG) as an effective solution to achieve image-pixel adaptation and to solve open-vocabulary semantic segmentation. Concretely, MVP-SEG deliberately learns multiple prompts trained by our Orthogonal Constraint Loss (OCLoss), by which each prompt is supervised to exploit CLIP feature on different object parts, and collaborative segmentation masks generated by all prompts promote better segmentation. Moreover, MVP-SEG introduces Global Prompt Refining (GPR) to further eliminate class-wise segmentation noise. Experiments show that the multi-view prompts learned from seen categories have strong generalization to unseen categories, and MVP-SEG+ which combines the knowledge transfer stage significantly outperforms previous methods on several benchmarks. Moreover, qualitative results justify that MVP-SEG does lead to better focus on different local parts.
Pixel-Level Sonar Image JND Based on Inexact Supervised Learning

The Just Noticeable Difference (JND) model aims to identify perceptual redundancies in images by simulating the perception of the Human Visual System (HVS). Exploring the JND of sonar images is important for the study of their visual properties and related applications. However, there is still room for improvement in performance of existing JND models designed for Natural Scene Images (NSIs), and the characteristics of sonar images are not sufficiently considered by them. On the other hand, there are significant challenges in constructing a densely labeled pixel-level JND dataset. To tackle these issues, we proposed a pixel-level JND model based on inexact supervised learning. A perceptually lossy/lossless predictor was first pre-trained on a coarse-grained picture-level JND dataset. This predictor can guide the unsupervised generator to produce an image that is perceptually lossless compared to the original image. Then we designed a loss function to ensure that the generated image is perceptually lossless and maximally different from the original image. Experimental results show that our model outperforms current models.
Anime Sketch Coloring Based on Self-attention Gate and Progressive PatchGAN

Traditional manual coloring methods require hand-drawn colors to create visually pleasing color combinations, which is both time-consuming and laborious. Reference-based line art coloring is a challenging task in computer vision. However, existing reference-based methods often struggle to generate visually appealing coloring images because sketch images lack texture and training data. To address this, we propose a new sketch coloring network based on the PatchGAN architecture. First, we propose a new self-attention gate (SAG) to effectively and correctly identifying the line semantic information from shallow to deep layers in the CNN. Second, we propose a new Progressive PatchGAN (PPGAN) to help train the discriminator to better distinguish real anime images. Our experiments show that compared to existing methods, our approach demonstrates significant improvements in some benchmark tests, with Fréchet Inception Distance (FID) improved up to 24.195% and Structural Similarity Index Measure (SSIM) improved up to 14.30% compared to the best values.
Adaptive Low-Light Image Enhancement Optimization Framework with Algorithm Unrolling

Images captured in a dark environment may cause low visibility and lose significant details leading to poor performance of vision-based recognition systems. Recently, deep learning-based methods have been proposed for low-light image enhancement (LIE) with different priors or training schemes. However, even those LIE methods may introduce visual artifacts into the enhanced images. This paper proposes an adaptive LIE optimization framework that allows to re-optimize different deep learning-based LIE methods based on an adaptive quality evaluation (QE). Specifically, we design an interpretable and learnable LIE-QE module for LIE. To find the optimal structure of the LIE-QE module, we propose an algorithm unrolling method to design the LIE-QE module, where the each layer of the decomposition component of the LIE-QE module can be interpreted as WLS edge-aware smoothing. Both qualitative and quantitative experiments were conducted, and the evaluation verified the effectiveness of the proposed learnable deep unrolled LIE-QE module for LIE. The results shows that the proposed LIE framework can effectively improve different deep learning-based LIE methods indicating the potential of the optimization framework with LIE-QE module to re-optimize existing DNN-based LIE methods.
Few-Shot Object Detection Algorithm Based on Adaptive Relation Distillation

Deep learning methods have advanced the accuracy and speed of object detection. However, acquiring labeled data is especially challenging in real-world scenarios. As a result, the general-purpose object detection algorithms based on deep learning experience significant performance degradation with limited samples. To tackle the issue of declining detection accuracy in the presence of few-shot data, this paper introduces a few-shot object detection algorithm based on adaptive relation distillation, which improves upon existing algorithms by enhancing the fusion of query and support features. In the proposed method, the adaptive relational distillation module discards the hand-designed and inefficient query information utilization strategy employed in previous algorithms and adaptively fuses the features of support and query images using convolutional networks. To augment the learning capability of the adaptive relational distillation module, we utilize a hybrid attention module in the support branch to emphasize the regions crucial for detecting specific classes of objects. The experimental results demonstrate our proposed algorithm achieves an average accuracy of 47.6% on three data divisions and five sample size settings for the PASCAL VOC dataset, which marks an improvement of 8.3% over DCNet.
Robust State Estimation Using the Maximum Correntropy Cubature Kalman Filter with Adaptive Cauchy-Kernel Size

The maximum correntropy criterion (MCC), as an effective method for dealing with anomalous measurement noise, is widely applied in the design of filters. However, its performance largely depends on the proper setting of the kernel bandwidth, and currently, there is no efficient adaptive kernel adjustment mechanism. To deal with this issue, a new adaptive Cauchy-kernel maximum correntropy cubature Kalman filter (ACKMC-CKF) is proposed. This algorithm constructs adaptive factors for each dimension of the measurement system and establishes an entropy matrix with adaptive kernel sizes, enabling targeted handling of specific anomalies. Through simulation experiments in target tracking, the performance of the proposed algorithm was comprehensively validated. The results show that the ACKMC-CKF, through its flexible kernel adaptive mechanism, can effectively handle various types of anomalies. Not only does the algorithm demonstrate excellent reliability, but it also has low sensitivity to parameter settings, making it more broadly applicable in a variety of practical application scenarios.
Prompt Based Lifelong Person Re-identification

In the real world, training data for person re-identification (ReID) comes in streams and the domain distribution may be inconsistent, which requires the model to incrementally learn new knowledge without forgetting the old knowledge. The problem is known as lifelong person re-identification (LReID). Previous work has focused more on the acquisition of task-irrelevant knowledge and neglected the auxiliary role of task-relevant information in alleviating catastrophic forgetting. To alleviating forgetting and improving the generalization ability, we introduced the prompt to learn task-relevant information, which can guide the model to perform task conditionally. We also proposed a special distillation module for the specific vision transformer structure, which further mitigated catastrophic forgetting. Extensive experiments on twelve person re-identification datasets outperforms other state-of-the-art competitors by a margin of 4.7% average mAP in anti-forgetting evaluation and 7.1% average mAP in generalising evaluation.