Reviewerly/exHarmony huggingface dataset:
Exploring affective image representation with visual attention and aesthetic fusion

Affective image analysis aims to understand the sentiment of different images. The challenge is to develop a discriminative representation that bridges the affective gap between low-level features and high-level emotions. Most existing studies bridge the gap by designing deep models carefully to learn global representations in one shot directly or identify image emotion by extracting features at different levels in the model. They ignore that both local regions of an image and relationships between them impact emotional representation learning. This paper develops an affective image analysis method based on the aesthetic fusion hybrid attention network (AFHA). A modular hybrid attention block is designed to extract image emotion features and model long-range dependencies of images. By stacking hybrid attention blocks in ResNet-style, we obtain an affective representation backbone. Furthermore, considering that image emotion is inseparable from aesthetics, we employ a modified ResNet to extract image aesthetics. Finally, through a fusion strategy, the image's emotion is considered with the aesthetics conveyed. Experiments demonstrate the close relationship between emotion and aesthetics, and our plan has an excellent competitive effect compared with existing methods on the image sentiment analysis dataset.
V-channel adaptive defogging with low Illumination images based on optimized retinex model

The defogging effects of different retinex algorithms, including single-scale retinex (SSR), multiscale retinex (MSR) and multiscale retinex with color restoration (MSRCR), are compared in this paper. It was found that some images treated by the above methods were dark. This phenomenon is more obvious when processing foggy images with low light. Contrast limited adaptive histogram equalization (CLAHE) is applied to improve the image brightness and contrast. Further analysis of the V-channel in HSV space shows that when the normalized histogram distribution of the V-channel is concentrated below 0.5 and the image has the component of the highlighted region, the images are dark after processing by the traditional retinex algorithms. Based on this, the V-channel adaptive enhancement method is proposed to improve the overall image brightness. The experiments show that the proposed algorithm works better when combined with both the modified MSR algorithm and CLAHE. The overall brightness of the image is improved, and the information entropy of the image is also increased.
A portrait image recommendation method based on collaborative filtering

Portrait recognition is a key task in public security area. Most present portrait recognition methods lay more emphasis on extracting facial features by different algorithms for a variety of scenarios, but focus less on fusion of multiple recognition results. In this paper, we propose a portrait image recommendation method based on collaborative filtering, for which the main task is to improve the hit accuracy by utilizing the portrait recognition results from multiple feature-based models. Considering these models preferences and historical scores on portrait images, user-based collaborative filtering is applied to calculate the similarity between the models. Meanwhile, due to the difference of similarity criteria, an improved comprehensive similarity model based on Auto-Encoder is established to synthesize similarity values calculated by different feature-based models to predict the rating scores of the candidate portrait images. Experiments on portrait images in the wild show that the hit accuracy reaches 85.48% and is 1.11% higher than that best results of the single feature-based models, which illustrates that the proposed method is effective and practical for recommendation of portrait images.
Unsupervised person re-identification based on intermediate domains

Unsupervised domain adaptive person re-identification (UDA re-ID) aims to transfer knowledge learned from the labeled source domain to the unlabeled target domain. Most recent methods focus on narrowing down the domain gap between the source and target domains while ignore the bridge between them. In this work, we explicitly model appropriate intermediate domains and construct two adaptation pairs (“source-intermediate” and “intermediate-target”) instead of the original “source-target” one pair adaptation. The purpose is to ease the adaptation difficulties caused by large domain gaps, making the adaptation process more smooth. To generate the intermediate domain, we use image-to-image translation methods which generate images that have the same contents and ID labels shared with the source domain and similar style to the target domain. When evaluated on standard benchmarks, our proposed methods outperforms the state of the arts by a large margin on the target domains where mAP of our method is higher than IDM [12] by 1.4% and 2.2% when testing on Market-1501 and MSMT17 respectively.
Application of augmented reality technology in train maintenance training system

Aiming at the lack of low-cost visual simulation tools for train maintenance, which can realize the failure simulation of various parts of the train, a train maintenance training simulation system based on augmented reality technology is proposed. Firstly, the overall design scheme of the train maintenance training simulation system is introduced. Secondly, using Unity3D as the driving engine, the stability of the virtual model display and the interaction with the virtual model during the maintenance process are designed. The Eazy AR interface and image processing technology are used to realize the roaming of the train maintenance scene, the fault maintenance of the train equipment and the corresponding human-computer interaction function are given, and the experimental results are given.
Episode-based training strategy for zero-shot semantic segmentation

We introduce episode-based training into zero-shot semantic segmentation (ZS3) for the first time. In particular, the model is trained on a set of simulated ZS3 tasks. The model gains the ability to predict simulated unseen classes over multiple episodes, which generalizes well to true unseen classes after training on multiple episodes. On the basis of this training framework, we propose a visual semantic alignment network named VSAN as our basic model, which mainly includes a feature extractor and semantic projection network. This base model constrains the visual-semantic distribution of the same class by a distance measure between visual features and semantic prototypes. In the inference phase, the trained projection network can generate corresponding semantic prototypes for all classes, and predict the segmentation results of the entire image by measuring the distance between visual features and semantic prototypes. The base model VSAN is called EB-VSAN after using the episode-based training strategy. Our model is a discriminative model, as opposed to generative methods, our model does not need retraining the classifier when a new class emerges, while avoiding multi-stage training. Our extensive ZS3 experiments on the benchmark dataset show that the EB-VSAN model outperforms current state-of-the-art methods, specifically, our hIoU metric outperforms state-of-the-art methods by an average of 1.7% on PASCAL VOC, while Average 2.5% improvement on PASCAL Context.
Semantic segmentation of high spatial resolution remote sensing imagery based on weighted attention U-Net

In recent years, with the development of deep learning and attention mechanism, more research has been carried out to realize semantic image segmentation based on deep learning integrated attention mechanisms. However, the current semantic segmentation methods have low segmentation accuracy, high computation cost, and serious loss of detailed information. In this paper, a lightweight designed attention gate model was introduced to reduce the computation cost. And because it can suppress irrelevant regions in the input image, while highlighting the salient features of specific tasks, the combination of the two weighting factors input features (<i> 𝑥<sup>𝑙</sup></i> ) and gating signal (g) in this structure can improve segmentation accuracy and reduce loss of detail. Therefore, this study used the weighted attention U-Net network to perform semantic segmentation on the GID dataset and finally evaluated it on the four indicators of Precision, Recall, F1-Sorce, and mIoU. This result shows that different weight values have a more significant impact on the experimental results. The attention U-Net with the best weight combination compared with the traditional U-Net network, Precision, Recall, F1-Sorce, and mIoU are increased by 0.88%, 1.4%, 1.13%, and 1.2%, respectively. Compared with the original attention UNet, Precision, Recall, F1-Sorce, and mIoU are increased by 0.86%, 1.24%, 1.04%, and 1.75%, respectively.
An improved YOLOX for remote sensing image object detection

Recent studies have shown that the attention mechanism can further improve the detection accuracy of YOLOX algorithm in remote sensing images, and coordinate attention can well solve the long-range dependencies problem of the previous attention modules, but its attention weights is too redundant in the channel dimension. At the same time, there is a problem of example imbalance in the training of YOLOX algorithm for remote sensing image object detection. To solve the above problems, an improved YOLOX algorithm is proposed, which combines the improved coordinate attention and focal loss. The former not only further adopts pooling and convolution operations to make the attention weights no longer contain redundant channel information when it still has the potential to capture long-range dependencies, but also introduces 1D convolution layers to obtain the final attention weights in three different directions to make the model pay more attention to the effective parts of the features of remote sensing images. The latter optimizes the quality of the gradients, which makes the training more effective and improves the detection accuracy. Training and testing with open remote sensing image dataset. The detection results show the effectiveness and superiority of our method.
Prior-guided dense up-down sampling network for face super-resolution and recognition

Identifying low-resolution (LR) face images in the wild is still an open and challenging task, 1as discriminative feature is hard to learn from LR image with blurry appearance. To address this problem, many face super-resolution (SR) methods have been proposed to reconstruct high-resolution (HR) face images from the observed LR images. However, most of existing methods have two drawbacks: they tend to generate over-smoothed results, and they largely ignore to recover the facial identity information. In this paper, we propose a new multi-task framework for very small face image SR and recognition. First, we propose the Dense Up-down Sampling Unit (DUSU), which is able to effectively represent the nonlinear LR-to-HR mapping via an error-correcting feedback mechanism. Secondly, we introduce an effective and robust facial prior knowledge, the high frequency sub-bands of Non-Subsampled Contourlet Transform (NSCT), to enhance the texture details of super-resolved images. Third, we introduce an evaluation network with perceptual loss to recover the identity information of reconstructed face images. Extensive experiments demonstrate that our method not only achieves more appealing results than the state-of-the-art methods in terms of traditional SR metrics, but also significantly improve recognition accuracy of very small faces in the wild.
Underwater stereo matching based on multilevel recurrent field transforms with iterative attentional feature fusion

Stereo matching for depth estimation is a fundamental vision problem. Recent works focus on deep learning to improve accuracy, but most networks encountered the difficulty of poor generalization ability and high computational cost especially on high resolution images. RAFTStereo achieves great advantages in these two aspects, but still can be improved further. In this paper, we revise the residual block of RAFT-Stereo in its feature extractors to improve the performance in underwater scenarios. Specifically, we choose an iterative Attentional Feature Fusion module to utilize the global information in feature fusion. To justify our work, we test our networks on ETH3D benchmark and our own underwater dataset, which demonstrates the superiority of our model as compared to the state-of-the-art baselines. Eventually, comparing to original RAFT-Stereo, our results on ETH3D benchmark outperform by 13.1% on the default metric bad 1-pixel error (percentage of pixels with end-point-errors greater than 1px) and results on our underwater dataset reduce the average error by 16.9%.
Effects of hyper-parameters setting in Bi-LSTM-CRF on Chinese named entity recognition

Named entity recognition is a basic task in the field of natural language processing. It also plays a role that can not be underestimated in the era of big data. This experiment will use the Bi-LSTM-CRF model to extract information from the input text to achieve the function of named entity recognition. In this experiment, we first select a suitable data set and perform vectorization, then build and train the Bi-LSTM-CRF model. At the same time, the dropout mechanism is added to assist. The optimal hyper-parameters will be found by constantly changing the parameter settings, so that the model shows the accuracy and robustness in the NER task. Each evaluation index reaches the optimal value. After the optimized model is obtained, the visualization of the model is carried out. All the entity parts are extracted from the input text and then output, showing the effect of the named entity recognition of the model and realizing a high level of named entity recognition.
Frequency domain deepfake detection based on two-stream neural network

Benefiting from the progress of deep learning driven generative models, face forgery technologies have rapidly become mature, thus raising public concerns about the illegal usage of these technologies. Despite the fact that fake images and videos are often unrecognizable to human eyes, recent work has found that hidden artifacts can be exposed in frequency domain. Our meth-od introduces the idea of separating human face area using landmarks before mining the forgery patterns in frequency domain. This we believe can help the network learn more discriminative features, also, a two-stream learning framework combining single-frame pathway and multi-frame pathway is developed to mine frequency clues. Compared with previous methods, our approach showed good results while using a few training data and little time. The effectiveness of our method is shown on different versions of the FF++ and Celeb-DF dataset.
A 3D self-adjustable organ aware deep network for abdominal segmentation in CT images

CT scan is an important reference means of disease diagnosis. Automatic segmentation of organ regions can save a lot of time and labor costs, and allow doctors to produce more intuitive observations of the internal organization of the human body structure. However, automatic multi-organ segmentation remains challenging due to the complicated anatomical structures and low tissue contrast in CT images. Traditional segmentation methods are relatively inefficient for organ segmentation and the traditional network architectures are rarely designed to meet the requirements of lightweight and efficient clinical practice. In this paper, we propose a novel network named Self-Adjustable Organ Attention U-Net (SOA-Net) to overcome these limitations. The SOA-Net includes multi-branches feature attention (MBFA) module and the feature attention aggregation (FAA) module. These two modules have multiple branches with different kernel sizes to adaptively adjust their receptive field size based on multiple scales of the target organs. The SOA-Net is a 3D self-adjustable organ aware deep network which can adaptively adjust receptive fields sizes based on multiple scales of the target organs to realize the efficient segmentation of multiple abdominal organs. We evaluate our method on a challenging abdominal multi-organ CT dataset, and the final experiments proved that our model achieves the best segmentation performance compared with the state-of-the-art segmentation networks.
Person re-identification with IBN layer and channel attention module for indoor scenarios

Person re-identification technology is being utilized increasingly frequently in autonomous processing and analysis of surveillance video jobs as a result of recent advancements in deep learning, particularly with safety precautions and smart transportation. As a result of the issues with inadequate illumination and reflection in indoor settings, At the moment, much of the related research on human re-identification concentrates on outside situations, with little attention paid to indoor scenarios. These make the process of person re-identification in complicated indoor scenarios very difficult. The indoor person re-identification algorithm is investigated in this research in order to increase the precision of person recognition in indoor settings. The IBN layer is an addition to the Resnet50 backbone network that uses a combination of instance normalization (IN) and batch normalization (BN) to eliminate individual appearance difference while retaining the feature difference of different individuals to address the issues with the obvious difference of light and shade in person images taken by indoor monitoring. To enhance the expressiveness capability of individual features, the attention module based on feature channel is added to the residual network. In specifically, the learning approach automatically determines the value of each channel in a person's attributes in order to amplify the important qualities and suppress the unnecessary ones. On the other hand, to address the issue of it being challenging to distinguish between similar people caused by more interference factors such as occlusion and reflection in indoor scene, we introduce triple loss in the model training process, which can make the model better learn the details of persons. The three primary validation data sets utilized in this study are Market1501, OUC365, and DukeMTMC-reID. The indoor style and high definition in the OUC365 data set are more obvious, the noise is more obvious in the Market1501 data set, and there is a significant difference in the number of photos among various people in the DukeMTMC-reID data set. The proposed method is tested on several data sets in this paper, and successful results are obtained.
Single image snow removal via multi-scale dual domain decomposition and fusion

Under snowy weather conditions, cameras are prone to the interference of snow and can severely reduce the quality of the captured images, which will affect the computer vision performance greatly. Since no temporal information can be exploited, snow removal from single image is a challenging problem. In this paper, a novel snow removal method from single image was proposed by designing a kind of multi-scale image processing framework both in the spatial and frequency domain. Firstly, the input snowy image was decomposed into detailed sub-images and approximate parts by the Laplacian pyramid transform. Secondly, the approximate part is decomposed again into the background and detailed sub-image by the edge-preserving and structure-preserving image smoothing filter. After that, the non-subsampled shearlet transform was introduced to detect snowflakes within the frequency domain of the detailed sub-images, while mathematical morphological filtering was adopted to remove the labeled snowflakes within their spatial domain. Finally, the desnowing image was obtained by the inverse Laplacian pyramid transform. Experiments on real-world snowy images show that the proposed method produces better results than those of other state-of-the-art methods.
Self-perceptual generative adversarial network for synthetic aperture sonar image generation

Due to the shortage of Synthetic Aperture Sonar (SAS) image datasets, the development of many underwater tasks is hindered. To tackle this problem, coupling optical rendering and image-to-image translation is a novel and feasible way. However, because of the big gap between simulated optical images and real SAS images, the performances of existing works are not desired and have plenty of room for improvement. In this paper, we introduce a Self-Perceptual Generative Adversarial Network (SPerGAN) which can controllably generate SAS images with high fidelity. It utilizes a kind of self-perceptual loss to generate high-quality and diverse SAS images. Moreover, we introduce a novel evaluation method of SAS images that accords closely with human cognition. To evaluate the performance of our method, we first compare it against recent outstanding image-to-image translation methods on qualitative and quantitative aspects. Then we make ablation studies to explore the effects of different cycle consistency losses and hyper-parameters. The results show that our method can generate diverse and realistic SAS images.
Low-rank and spectral-spatial variation regularized hyperspectral image denoising algorithm

Hyperspectral image (HSI) is an important carrier for recording, transmitting, and storing information. Denoising is an indispensable step in HSI processing, which is the process of restoring noisy images to high-quality images that can reflect the objective world. It is crucial to establish a reasonable denoising method. Most recent studies regard HSI as a three-dimensional tensor and then establish the low-rank method, or use the spatial information of the image to establish a total variation method. In this paper, we combine the spectral-spatial structure and low-rank properties of HSI to construct the regularization term and propose a low-rank spectral-spatial adaptive total variation (LRSSAHTV) model. Then we define a separable soft thresholding (SST) operator to minimize the spectral-spatial adaptive total variation (SSAHTV) regularization problem, which is the basic theory of our algorithm. A linearized alternating direction method of multipliers (LADMM) algorithm is proposed to solve the composite model. To verify the effectiveness of our method, we perform the simulated noisy HSI and the real polluted HSI denoising experiments in Section 4. We also compare the denoising effect of the proposed method with the total variation method and low-rank method, which shows that our method can obtain the most precise restored image.
A lightweight convolutional network based on pruning algorithm for YOLO

With the rapid development of deep learning, neural network models have become increasingly complicated, leading to larger storage space requirements and slower reasoning speed. These factors make it difficult to be deployed on resourcelimited platforms. To alleviate this problem, network pruning, an effective model compression method, is commonly performed in a deep neural network. However, traditional pruning methods simply set redundant weights to zero, thus failing to achieve the acceleration effect. In this paper, a channel-wise model scaling method is proposed to reduce the model size and speed up reasoning by structurally removing the redundant filters in convolutional layers. To make the residual block more sparse, we develop a pruning method for residual cells. Experimental results on the YOLOv3 detector show that our proposed approach achieves a 70.6% parameter compression ratio without compromising accuracy.