Reviewerly/exHarmony huggingface dataset:
Adaptive locality sensitive analysis representation learning via K-SVD algorithm

Recent years, analysis representation learning and its applications for classification have been well explored and applied, due to its flexible representation ability and low classification complexity. With a learned analysis dictionary, test samples can be transformed into a sparse subspace for classification efficiently. However, the underlying locality of sample data has rarely been explored and reliably married with analysis representation learning to enhance the discriminative capability of the classifier. In this paper, we propose a novel adaptive locality-sensitive analysis representation learning model for pattern classification (ALAR). It considers the intrinsic geometric properties by imposing adaptive weighted constrained graph regularization to uncover the geometric structure of the image data. Through the learned analysis dictionary, we transform the image to a new and compact space where the manifold assumption can be further guaranteed. Thus, the local geometrical structure of images can be preserved in sparse representation coefficients. Moreover, the ALAR model is iteratively solved by the synthesis K-SVD and gradient technique. Experimental results on image classification validate the performance superiority of our ALAR model.
Siamese network algorithm based on multi-scale channel attention fusion and multi-scale depth-wise cross correlation

The research takes the feature extraction network and depth-wise cross correlation learning method of the Siamese network as the starting point. Firstly, the regression strategy of the proposed framework is anchor-free, and the residual network ResNet50 is chosen as the backbone network, and add the channel attention mechanism SENet. The SEMSCAM multi-scale channel attention model is proposed to make up for the lack of local feature extraction ability of the feature extraction network on the basis of SENet. On this basis, the attention fusion module AFFN is added to enhance the soft selection of attention. Combined with the SE-MSCAM multi-scale attention model and the attention fusion module AFFN, the ResNet50-AFFN multi-scale channel attention fusion network is proposed. Secondly, regarding the limitation of single-scale learning of SiamRPN++ depth-wise cross correlation, the MS-DWXCorr multi-scale depthwise cross correlation is proposed which increases the diversity of learning feature scales to improve the efficiency of tracking network similarity learning. The experimental results show that, on the VOT2018 benchmark, the EAO of our method outperforms 4.0% of the mainstream algorithm SiamCAR, the tracking accuracy is improved by 3.4% and the tracking speed of our method maintains 40 FPS; the tracking success rate is improved by 2.0% and the tracking accuracy rate is improved by 3.2% compared to the mainstream algorithm SiamCAR. It has higher accuracy and robustness in dealing with occlusion, deformation, illumination variation, deformation, and other scenarios of visual tracking, and has better tracking performance.
DRA-Net: densely residual attention based low-light image enhancement

The visual quality of nighttime photographs diminishes greatly due to low contrast and high noise. We need a robust image enhancement methodology to improve such low-light images close to standard daylight images. Due to deteriorated conditions of uneven light and noise, this image enhancement problem becomes ill-posed. Our paper has proposed a Densely Residual Attention Network (DRANet), an end-to-end attention base densely residual network. The architecture of DRANet consists of sub-modules convolution block (CB) and densely residual feature - convolutional block attention module (DRF-CBAM). DFR-CBAM also has sub-modules, deep residual feature block (DRFB), and convolutional block attention module (CBAM). Using the most recent results from attention and deep residual-based convolution networks in number of computer vision problems, we have used DRFB to enhance the features in-depth by using its dense and residual skip connections. Similarly, features in both spatial and channel axis have been extracted by using a lightweight CBAM attention module. Contrast, luminosity, and noise of the enhanced images have been balanced by additionally implementing a color balancing function at the end of the proposed network. Furthermore, we have used a combination of <i>L<sub>Lab</sub></i>, <i>L<sub>SSIM</sub></i> and <i>L<sub>MAE</sub></i> loss functions to make the proposed network stable and recover both contextual and local details while training. MAE, PSNR, SSIM, MS-SSIM, FSIM, Cosine Similarity, and deltaE2000 have been used as referenced while NIQE as nonreferenced based image quality assessment (IQA) metric. Experiment results showed that our proposed methodology is very effective with higher referenced and lower non-referenced image IQA metrics values. Furthermore, the effectiveness of our method has also demonstrated by the visual and perceptual quality of enhanced images.
Semantic segmentation of road scene based on the mechanism of adversarial attention

Semantic segmentation plays an increasingly important role in the field of computer vision. The current semantic segmentation algorithms are mainly based on full convolutional network downsampling for feature extraction, however, the reduction of image resolution due to downsampling will inevitably lead to the loss of image information and the process is irreversible. Although features can be reused to compensate for the loss, the problem of lost pixel correlation and inaccurate segmentation results still occur. In order to solve these problems, the attention mechanism and the generative adversarial network are fused into the semantic segmentation network, and a semantic segmentation network is proposed. The attention mechanism is added to the generator and discriminator modules and trained successively, then the generator module with rich information and global information is extracted, and its output is fused with the feature graph sampled above, in order to reduce the loss of local information and complete the task of image segmentation, an interpolation method based on image edge is introduced to sample the image, which improves the bilinear interpolation edge blur. Experiments on Cityscapes dataset show that the proposed network model is effective and reliable.
Object detection in infrared images using modified YOLOv4 models and an image enhancement module

Deep learning-based object detection approaches have shown excellent performance in RGB images. However, when used to detect objects from infrared images, the accuracy may reduce significantly due to low contrast, obscure textures and strong noise of infrared images. To alleviate the problem, we design a detail enhancement module involving spatial attention mechanism to enhance the textures and details of images. The output of the proposed module is fed into modified YOLOv4. We introduce Alpha-IoU loss and Weighted-NMS to YOLOv4 to enhance geometric factors in both bounding box regression and Non-Maximum Suppression, leading to notable gains of average precision. The experiment results show that compared with YOLOv4, mAP0.5 and mAP0.5:0.95 of our model are improved by 1.1% and 3.5% respectively, effectively improving the detection accuracy.
Instance-level image synthesis method based on multi-scale style transformation

Semantic image synthesis is to synthesize photorealistic images according to the given semantic layout. Existing methods try to build a single-scale style encoder based on semantic regions, which inject style simply based on a single level, are unable to extract rich style information. Especially for different instance objects in the same semantic region, single-scale networks tend to generate the same style and control style ineffectively. To cope with this issue, we propose Multi-Scale Instance-level image synthesis method (MSIN). In order to learn more discriminative representation from different feature levels in instance, a multi-scale style encoder is designed to extract more details instead of traditional single-scale style encoder, which adopts a "pyramid" structure to contact contextual information. In addition, to synthesize visually pleasing and photorealistic images, MSIN leverages the region-style fusion mechanism in adaptive normalization layer, which realizes instance-wise object-to-object multi-style generation simultaneously. Compared with the previous methods, our method can generate images with fine details and control style in instance object, whose semantics are more reasonable and diverse to different instance objects. The experimental results demonstrate the superiority of MSIN on dealing with semantic image synthesis tasks and outperforms existing methods in terms of instance objects and diverse generation.
A non-local image denoising method based on TV-L1 with variable exponents

Among various image denoising methods, the total variation (TV) has offered considerable performance. However, it is prone to the staircasing effect. To alleviate this problem, we propose a novel TV-based denoising method that utilizes variable exponents as the regularization constraints and L1-norm as the fidelity term. To further improve the robustness of our method, we develop it under a non-local framework to avoid overlooking the global information of the image. Extensive experiments on real-world images have validated the effectiveness of the proposed method.
A novel image scaling algorithm based on wavelet transform and polyphase filtering

Due to the poor performance of the traditional interpolation algorithms in terms of the video image scaling effect and the low algorithm complexity required by existing hardware implementations, a scaling algorithm based on wavelet reconstruction and polyphase filtering implementation is proposed. The method first extracts the pre-generated weight coefficients according to the scaling requirements that including scaled image quality, size, and processing speed. Furthermore, the filter coefficients are interpolated with the source image pixel points mapped by the target image through a polyphase filter. Moreover, the resulting interpolated image is selectively decomposed and reconstructed by Haar wavelet transform to generate super-resolution images. The experimental results show that the algorithm improves the scaling quality of the images both subjectively and objectively, while enhancing the processing speed of the images and achieving non-integer scaling.
Classification of breast cancer pathological images combining fine-grained region location

Classification algorithm based on deep learning is the main technology for computer-aided intelligent diagnosis of breast pathological images. The existing deep learning algorithms rarely pay attention to multi-scale information in breast cancer pathological images, and can not extract key regions from pathological images for auxiliary diagnosis. To address this issue, this paper proposes a novel method for classifying breast cancer pathological images which incorporates a fine-grained region location mechanism. This method is realized by a dual-branch architecture with global network and local network. Global and local features are extracted from the whole image and local key image, respectively. The final classification results can be obtained by integrating both the global and local network analysis. In order to locate the most predictive area in the whole image, this paper designs utilizes a fine-grained region localization mechanism and combines the above two branch networks. Extensive experiments on the BreakHis data set are conducted to verify the effectiveness of the proposed algorithm. The empirical results show that this method improves the classification accuracy by comparing the performance with that of several typical convolutional neural network and state-of-the-art algorithms.
Crowded people detection for occluded classroom surveillance scenes based on relation model

People detection is an important task in video surveillance. Due to the people’s similar characteristics and occlusion, crowded people detection for occluded classroom surveillance scenes is challenging. In this paper, a new detection framework based on the relation model method is proposed to detect crowded people in occluded classroom surveillance scenes. Our method is mainly to predict a box set of related objects and then use the positive boxes to refine the noisy boxes. Specifically, a new box set selector is designed to select positive boxes prone to generating accurate predictions, and then the rest occluded boxes are refined through the relation model module. To demonstrate the effectiveness of our proposed method, a new classroom video surveillance dataset ICDU is made, and we conduct extensive experiments on this classroom video surveillance dataset and the public dataset CrowdHuman. Experiment results show that our proposed method performs excellently on our ICDU dataset and CrowdHuman dataset.
Fast line segment matching based on point-line affine invariants and pairwise constraints

In this paper we propose a method about line segment matching based on point-line invariants. We use ORB and EDlines, two efficient and stable methods, to extract point and line features respectively. Then we introduce the implementation details of our matching algorithm. It includes affine invariance of the ratio of the distance from two coplanar points to the line and pairwise constraint of the geometric relationship between two lines. In order to eliminate mismatches, we use a series of methods to optimize the result of our algorithm. We set up a scoring mechanism for candidate matches and the final matches will be given by evaluating the voting matrix. Its performance is evaluated by extensive experiments. The results show that our proposed method outperforms the mainstream methods, and are robust to rotation, scale, blur and other transformations.
Vehicle detection in the infra-red thermal images based on a sparse neural network

Recently, infra-red thermal images have drawn wide publicity for vehicle detection in traffic monitoring because of their resistances to insufficient light and hidden occasions. However, there are two significant challenges compared with vehicle detection in visible images. One is the insufficiency of the labelled infra-red thermal samples, and the other is less texture information. These two points lead to fewer parameters for the vehicle detection network in the infra-red thermal images. Therefore, to conquer these difficulties in infra-red thermal images, we construct a sparse neural network based vehicle detection method with sparse parameters (weights) and high generalization ability. Therein, to avoid over-fitting in the training stage, we eliminate some redundant parameters via constraints and on-line pruning. Finally, to verify the proposed vehicle detection method in infra-red thermal images, we introduce VIVID, VOT2019, and RGBT234 data sets. The vehicle detection results show that the proposed method presents superior results compared to several mainstream approaches.
Forgery Detection by Weighted Complementarity between Significant Invariance and Detail Enhancement

Generative adversarial networks have shown impressive results in the modeling of movies and games, but what if such powerful image generation capability is used to harm the Multimedia? The face replacement methods represented by Deepfakes are becoming a threat to everyone, so the development of image authenticity detection methods has become a top priority. For achieving accurate detection resistant to compression effects, we propose a weighted complementary dual-stream detection method. Firstly, to alleviate the influence of image compression on manipulation detection, we propose the concept of pixel-wise saliency invariance. We map fake images onto saliency maps via Quaternary Fourier Transform, which discovers the invariant properties of image phase spectra on different compressions. Meanwhile, to capture boundary traces more easily, we propose the concept of pixel-wise detail enhancement. We apply Bilateral Filtering to preserve the texture edges of fake images and amplify the fake boundaries. Finally, to take full advantage of the two proposed concepts, a weighted complementary dual-stream network (WCD Network) is designed as a classifier to fuse features and identify real and fake. On different benchmarks like FaceForensics++(FF++), Celeb-DF and DFDC, the experimental results show that the proposed method has the average best detection accuracy compared to existing methods.
Max-margin class incremental learning with mixup augmentation

The most famous dilemma for class incremental learning(CIL) is the catastrophic forgetting phenomenon, where the trained model quickly forgets to recognize the old classes when learning novel ones. To cope with this problem, we propose a novel max-margin class incremental learning with less forgetting, which can well discriminate the relationship between classes and prevent knowledge forgetting in the incremental learning process. Specifically, we introduce a maxmargin objective function to explicitly enforce the maximum distance between different classes larger than a predefined margin, which can avoid the ambiguity between old and new classes and then increase the discriminative ability of the learned classifier. Further, we adopt a mixup augmentation mechanism by performing the mixup operation on the reserved images of old classes and the incremental images of new classes, especially to reduce knowledge forgetting. Comprehensive evaluations and comparisons on three public datasets (including CIFAR-100, ImageNet-Subset and ImageNet) well demonstrate that our proposed CIL approach can effectively improve performance compared to the existing CIL methods.
Complementary features-aware attentive multi-adapter network for hyperspectral object tracking

Hyperspectral object tracking aims to estimate the bounding box for the given target using hyperspectral data. Different from traditional color videos, hyperspectral videos have more abundant band information for their capacity to capture the reflectance spectrum of the target at a wider range of wavelengths provides important capabilities and opportunities, which provides new capabilities for discriminating targets in complex scenes, but also presents new challenges. The limited dataset and the high dimensionality of hyperspectral data are two new challenges in constructing hyperspectral trackers, resulting in existing hyperspectral tracking methods based mainly on correlation filters. This paper proposes a new Complementary Features-aware Attentive Multi-Adapter Network (CFA-MANet), which can train a neural network well and achieve high performance for Hyperspectral Object tracking just using the limited dataset. Specifically, we add a complementary features-aware module to the multi-adapter network, which employs two different strategies to reduce the dimensionality of hyperspectral data from two complementary perspectives, and the joint implementation of these two strategies results in a reduction in the amount of computed data and parameters of the designed neural network while achieving competitive results. Moreover, spatial and channel attention modules are used to learn a wider range of contexts and improve the representation of different semantic features, respectively. Crossattention is used to learn complementary information and thus generate more discriminative representations. Experimental results on hyperspectral datasets show that our method achieves the best results compared to several recent hyperspectral tracking methods.
Garbage classification model integrating attention mechanism

In this paper, we propose a garbage classification model that integrates the attention mechanism and multiple network optimization methods. First, we construct a four-category primary network for recyclable garbage, kitchen garbage, hazardous garbage, and other garbage. And then, four secondary networks are constructed to map the above 4 primary categories to 40 secondary classes. Both the primary and secondary networks take Resnet101 as the main backbone network and integrate attention mechanism, Focal loss function, and warm-up learning rate. The experimental results prove that the proposed model has a high classification performance for the HUAWEI cloud garbage classification dataset.
Lightweight human pose estimation with attention mechanism

There are certain inevitable challenges in human pose estimation tasks based on deep learning methods, such as large amount of network parameters and high computational complexity. This paper proposes a lightweight network to reduce the scale of model parameters and computational complexity, meanwhile improve the accuracy of the human pose estimation task. The new method takes the high-resolution networks HRNet-32 as the basic framework and replaces the basic module with the MBConv lightweight module. The attention mechanism is incorporated into the network to model the context information, so as to improve the perception ability and the feature extraction ability of the module and improve the accuracy of human pose estimation. The experimental results on COCO2017 show that the proposed network can detect human key points with high precision even when the amount of parameters is reduced by 56%, which verifies that the proposed method has good lightweight performance.
Multi-scale transformer 3D plane recovery

The 3D plane recovery and reconstruction method of a single image aim to identify plane instance regions and estimate plane parameters and is widely used in technologies such as AR/VR(Augmented Reality/Virtual Reality). In recent years, significant progress has been achieved in single-image 3D plane recovery approaches based on deep learning, however much of the research has focused on overall plane segmentation performance rather than the accuracy of small-scale plane segmentation. Given the lack of an accurate definition of the segmentation of small-scale plane regions in existing methods, we propose a novel multi-scale transformer-based plane recognition and recovery model, which can accurately identify the edges of small-scale plane regions. The network branches composed of the model are used to detect plane and non-planar areas respectively. Different input feature scales make the two network branches have different global feature extraction capabilities. Finally, the two branches are strengthened to recognize plane areas in the same scene through mutual information loss. The consistency makes the two branch networks have the ability to share parameters. The experimental results in the Scannet and NYU V2 datasets show that the model can accurately identify small-sized plane areas, and the detection and recovery accuracy reaches the state-of-the-art effect.