Reviewerly/exHarmony huggingface dataset:
3D face alignment and face reconstruction based on image sequence

Existing 3D face alignment and face reconstruction methods mainly focus on the accuracy of the model. When the existing methods are applied to dynamic videos, the stability and accuracy are significantly reduced. To overcome this problem, we propose a novel regression framework that strikes a balance between accuracy and stability. First, on the basis of lightweight backbone, encoder-decoder structure is used to jointly learn expression details and detailed 3D face from video images to recover shape details and their relationship to facial expression, and dynamic regression of a small number of 3D face parameters, effectively improve the speed and accuracy. Secondly, in order to further improve the stability of face landmarks in video, a jitter loss function of multi-frame image joint learning is proposed to strengthen the correlation between frames and face landmarks in video, and reduce the difference amplitude of face landmarks between adjacent frames to reduce the jitter of face landmarks. Experiments on several challenging datasets verify the effectiveness of our method.
Multi-visual information fusion and aggregation for video action classification

In order to fully mine the performance improvement of spatio-temporal features in video action classification, we propose a multi-visual information fusion time sequence prediction network (MI-TPN) which based on the feature aggregation model ActionVLAD. The method includes three parts: multi-visual information fusion, time sequence feature modeling and spatiotemporal feature aggregation. In the multi-visual information fusion, the RGB features and optical flow features are combined, the visual context and action description details are fully considered. In time sequence feature modeling, the temporal relationship is modeled by LSTM to obtain the importance measurement between temporal description features. Finally, in feature aggregation, time step feature and spatiotemporal center attention mechanism are used to aggregate features and projected them into a common feature space. This method obtains good results on three commonly used comparative datasets UCF101, HMDB51 and Something.
FSC-UNet: a lightweight medical image segmentation algorithm fused with skip connections

In order to study the effect of skip connections to segmentation performance in encoder and decoder networks, in this paper, we improve the skip connections of U-Net model and adopt the method of sub-module fusion connection. We fuse the high and low layers of the encoder by multi-head attention. Fusion is performed separately, and the fusion result is connected to the decoder. Considering that different input images have different effects to model training due to factors such as noise, we set the threshold by calculating the Euclidean distance between the image and the mask during training, so that different images use different skip connection methods. Experiments on Cell nuclei, Synapse, Heart, Chaos datasets show that FSC-UNet algorithm this paper proposed has better results than existing algorithms.
Systematization of digitalization directions of higher education in Russia

Scientific and technological progress and digitalization stimulate the dynamic development of Internet programs for the development of higher education in Russia. Digitalization has affected all aspects of the educational process, including: management, communications, a paradigm shift in education and a complete transformation of the traditional school of higher education. Digital methods of teaching in higher education are integrated into existing traditional training programs, there is a digital update of educational programs and disciplines. Mixed learning models are being created, distance education programs are being developed and implemented, a digital educational environment with system unified modules is being introduced, a digital approach to managing educational institutions has been established.
Subcortical brain segmentation with convolutional neural networks

Subcortical brain segmentation is a challenging task due to the anatomical variability in both shape and size between patients, such as thalamus, hippocampus and amygdala. It requires the accurate segmentation of these structures to measure their volume and surface. However, few methods can obtain accurate segmentation because the boundaries of these structures are obscure in MR images. We propose an attention-based convolutional neural network for subcortical brain segmentation. In our method, image clipping is firstly applied for pre-processing. Accurate subcortical brain segmentation is obtained by using attention-based convolutional neural network. Maximum connectivity is finally applied for post-processing. Experimental results in 35 subjects showed that the proposed method segment the brain region with higher accuracy than other methods. The Dice, TPR and VD measures show that the proposed method is able to provide a precise and robust segmentation estimate. The proposed method is a suitable alternative to assist the manual subcortical brain segmentation task.
A sample diversity and identity consistency based cross-modality model for visible-infrared person re-identification

Visible-infrared person re-identification (VI-ReID) aims to search person images across cameras of different modalities, which can address the limitation of visible-based ReID in dark environments. It is a very challenging task, as images of the same identity have huge discrepancy in different modalities. To address this problem, a cross-modality ReID model based on sample diversity and identity consistency is proposed in this paper. For sample diversity, auxiliary images are introduced based on the idea of information transfer. The auxiliary images combine the information of visible images and infrared images, and can improve the diversity of input data and robustness of the network. For identity consistency, homogeneous distance loss and heterogeneous distance loss are developed from four different perspectives to shorten the distance between the samples of same identities. Extensive experimental results demonstrate the effectiveness of the proposed method.
Generative image inpainting with residual texture prior and cross-layer contextual attention

Existing image inpainting methods have shown promising performance in filling the missing regions with visually plausible contents. However, these methods tend to produce distorted structure and blurry texture. To address these issues, in this paper we propose a two-stage inpainting network that combines texture generation and image completion. In the first stage, a texture generator is used to hallucinate texture of the missing regions to guide the reconstruction in the next stage. In the second stage, considering the texture prior would gradually lose its guiding role with the deepening of the network, we adopt residual texture prior to generate fine details. We also introduce a cross-layer contextual attention module which can not only learn contextual attention in decoder feature map, but also benefit from the similar feature shifted from the encoder, generating reasonable structure and realistic texture. Our comparison results of both qualitative analysis and quantitative experiments on Paris StreetView and CelebA datasets demonstrate our proposed method has better inpainting performance than existing methods.
Dock detection method in remote sensing images based on improved YOLOv4

The dock target in remote sensing images has the characteristics of slender structure and direction arbitrarily. The general target detection algorithm based on the convolutional neural network cannot effectively obtain the direction information of the target, which cannot meet the actual demand of dock detection. This study designed a deep convolutional neural network architecture in any direction based on the YOLOv4 algorithm aimed at resolving the above problems. First, the multidimensional coordinate method was used to calibrate the dock target so that the network could contain the direction information of the target. Second, the loss function of the algorithm was optimized to make it suitable for directional target detection. Finally, an attention mechanism was introduced to enhance the extraction ability of the algorithm and further improve its detection accuracy. Two datasets of dock target detection from remote sensing images were selected for experiments, and the results showed that the improved YOLOv4 network was better than the other networks in the dock target detection task.
Deep learning video analytics solutions for ocean surveillance systems

Moored data buoys are floating platforms at sea. These buoys serve as in-situ Weather, Ocean and Tsunami observatories. These buoys transmit real-time data through 3G/GSM/GPRS and satellite telemetry. Damage to the buoy systems by humans, boats, ships etc., intentional or otherwise, causes loss of data, and inhibits early warning systems. It also has financial implications due to the loss of the instruments, repair & reinstallation charges, and the time a ship spends to fix the buoy. Challenges arise while analyzing the video footage as they are unstable and shaky, due to the continuous movements of floating ocean buoy platforms caused by the state of the sea. This paper explores object detection algorithms for detecting eight different objects commonly found in the camera video footage transmitted by the buoy platforms at sea. The object detection training implementation gave us a best accuracy of 0.867MAP@0.5IOU. The object detection will help in solutions like Object Search, detection of floating marine plastic debris, understanding the direction of motion of ships, boats etc. In a broader perspective, it can help in Surveillance, Market Survey and Fish Detection in underwater cameras for fish abundance study.
Correction of images projected on non-white surfaces based on deep neural network

When projecting onto a non-white surface, the projected image is distorted or color mixing by complex luminance and chrominance information, which makes the projection result different from the visual perception of the human eye. The purpose of projection image correction is to remove these effects, and traditional solutions usually estimate parameters from the collected projection samples, compute an inverse model of the projection imaging process, and try to fit a correction function. In this paper, a deep neural network-based projection image correction network (PICN) is designed to implicitly learn complex correction functions. PICN consists of a U-shaped backbone network, a convolutional neural network that extracts projected surface features, and a perceptual loss network that optimizes the correction results. Such a structure can not only extract the deep features and surface interference features of the projected image, but also make the corrected projected image more in line with human visual perception. In addition, we built a projector-camera system under the condition of a fixed global illumination environment for verification experiment, and proved the effectiveness of the proposed method by calculating the evaluation metrics of projected images before and after correction.
Identifying recaptured images using deep hybrid correlation network

With the explosive advancements in image display technology, recapturing high-quality images from high-fidelity LCD screens becomes more and more easy. Such recaptured images can not only be used for deceiving intelligent recognition systems but also for hiding tampering traces. In order to prevent such a security loophole, we propose a recaptured image detection approach based on deep hybrid correlation network. Specifically, we first design a deep hybrid correlation module to extract the correlations in different color channels and neighboring pixels. This module has three different branches, in which a 1×1 convolution layer is used to learn the correlations between color channels while two consecutive convolution sub-modules are used to extract the correlations between neighboring pixels. Then we feed the output of this module into consecutive convolution modules to further learn the hierarchical representation for make decision. Ablation experiments verify the effectiveness of our proposed deep hybrid correlation module, while single database experiments demonstrate that our proposed method can achieve average accuracy with about 99% on three public databases. Specifically, our method not only performs very close to the state-of-the-art methods on the most difficult-to-detect ICL-COMMSP database and the relative low-quality NTU-ROSE database, but also improves the performance on the most diverse Dartmouth database obviously, which verifies the effectiveness of the proposed deep architecture. Besides, mixed database experiments verify the superiority of the generalization ability of our proposed method.
A joint feature aggregation method for robust masked face recognition

Masked face recognition becomes an important issue of prevention and monitor in outbreak of COVID-19. Due to loss of facial features caused by masks, unmasked face recognition could not identify the specific person well. Current masked faces methods focus on local features from the unmasked regions or recover masked faces to fit standard face recognition models. These methods only focus on partial information of faces thus these features are not robust enough to deal with complex situations. To solve this problem, we propose a joint feature aggregation method for robust masked face recognition. Firstly, we design a multi-module feature extraction network to extract different features, including local module (LM), global module (GM), and recovery module (RM). Our method not only extracts global features from the original masked faces but also extracts local features from the unmasked area since it is a discriminative part of masked faces. Specially, we utilize a pretrained recovery model to recover masked faces and get some recovery features from the recovered faces. Finally, features from three modules are aggregated as a joint feature of masked faces. The joint feature enhances the feature representation of masked faces thus it is more discriminative and robust than that in previous methods. Experiments show that our method can achieve better performance than previous methods on LFW dataset.
Improved video classification method based on non-parametric attention combined with self-supervision

It is worth mentioning that in the video sequence modeling, the best recognition architecture is transformer. The current popular transformer based video classification methods focus on the importance of current features in time sequence. The degree of characterization of simultaneous order is insufficient, and simple data augmentation has unstable classification effect. In this paper we proposed a method of non-parametric attention combined with self-supervised feature construction to further improve video classification. In this method, the non-parametric attention mechanism is constructed in the simultaneous order feature to fit the multi-local extreme value distribution. At the same time, in the process of model learning, the input video is randomly masked in temporal domain and spatial domain, and self-supervised information is added to effectively learn the details and classification information of video content. Experiments using kinetics400, kinetics600 and something V2 datasets show that the algorithm in this paper has better improvement in accuracy than the current optimal method.
Deep learning techniques for image recognition of counterfeited luxury handbags materials

Due to the fact that counterfeit in second-handed goods terribly affects trading in markets of second-handed luxury bags, users in this research thus present studies of methods to classify genuineness of ‘Gucci GG Canvas’ with the pretrained model from Model VGG16 and with DenseNet121 to design deep Convolutional Neural Networks (CNN) model for binary classification. The CNN together with DenseNet121 model comprises accuracy at 95%, which is more than the 2 prior models, i.e., CNN from scratch and CNN together with VGG16.
An algorithm for facial mask area repair based upon deep learning

Due to the impact of Corona Virus Disease 2019 (COVID-19), facial mask has become a necessary protective measure for people going out in the last two years. One's mouth and nose are covered to suppress the spread of the virus, which brings a huge challenge for face verification. Whereas some existing image inpainting methods cannot repair the covered area well, which reduces the accuracy of face verification. In this paper, an algorithm is proposed to repair the area covered by facial mask to restore the identity information for face authentication. The proposed algorithm consists of an image inpainting network and a face verification network. Among them, in image inpainting network, to begin with, two discriminators, namely global discriminator and local discriminator. Then Resnet blocks are employed in two discriminators, which is used to retain more feature information. Experimental results show that the proposed method generates fewer artifacts and receives the higher Rank-1 accuracy than other methods in discussion.
Weakly-supervised attention mechanism via score-CAM for fine-grained visual classification

Along with the prosperity and development of computer vision technologies, fine-grained visual classification (FGVC) has now become an intriguing research field due to its broad application prospects. The major challenges of fine-grained classification are mainly two-fold: localization of discriminative region and extraction of fine-grained features. The attention mechanism is a common choice for current state-of-art (SOTA) methods in the FGVC that can significantly improve the performance of distinguishing among fine-grained categories. The attention module in different designs is utilized to capture the discriminative region, and region-based feature representation encodes subtle inter-class differences. However, the attention mechanism without proper supervision may not learn to provide informative guidance to the discriminative region, thus could be meaningless in the FGVC tasks that lack part annotations. We propose a weakly-supervised attention mechanism that integrates visual explanation methods to address confusing issues in the discriminative region localization caused by the absence of supervision and avoid labor-intensive bounding box/part annotations in the meanwhile. We employ Score-CAM, a novel post-hoc visual explanation method based on class activation mapping, to provide supervision and constrain the attention module. We conduct extensive experiments and show that the proposed method outperforms the current SOTA methods in three fine-grained classification tasks on CUB Birds, FGVC Aircraft, and Stanford Cars.
An effective anti-interference visual tracking method

Tracking specific objects in images or videos is one of the most attractive problems in visual tasks. It is widely employed in security monitoring, automatic driving, military operations and other scenes. Recently, object tracker based on convolution neural network, especially Siamese network, obtains high accuracy and has been deeply studied. However, in practical application scenarios of visual tracking, when meets clutter background or the object is occluded, the accuracy of the tracking task will drop rapidly, and the tracker loses the target in extreme cases. It is particularly necessary to quickly and accurately relocate the target. Therefore, an anti-interference tracker based on Siamese convolution neural network is developed. Benefiting from the adaptive tracking confidence parameter, once the tracking effect of the tracker has dropped significantly during the tracking process, the location of the object will be corrected immediately. Experimental results show that the proposed method has the ability to relocate and track the target after occlusion or loss effectively.
Compressing CNN by alternating constraint optimization framework

Tensor decomposition has been extensively studied for convolutional neural networks (CNN) model compression. However, the direct decomposition of an uncompressed model into low-rank form causes unavoidable approximation error due to the lack of low-rank property of a pre-trained model. In this manuscript, a CNN model compression method using alternating constraint optimization framework (ACOF) is proposed. Firstly, ACOF formulates tensor decomposition-based model compression as a constraint optimization problem with low tensor rank constraints. This optimization problem is then solved systematically in an iterative manner using alternating direction method of multipliers (ADMM). During the alternating process, the uncompressed model gradually exhibits low-rank tensor property, and then the approximation error in low-rank tensor decomposition can be negligible. Finally, a high-performance CNN compression network can be effectively obtained by SGD-based fine-tuning. Extensive experimental results on image classification show that ACOF produces the optimal compressed model with high performance and low computational complexity. Notably, ACOF compresses Resnet56 to 28% without accuracy drop, and the compressed model have 1.14% higher accuracy than learning-compression (LC) method.
Reversible data hiding simultaneously using substitution of MSB and compression of LSB of encrypted image

With the development of privacy protection, reversible data hiding methods in encrypted image have drawn extensive research interest. Among them, a new method is proposed based on embedding prediction errors, i.e., EPE-based method, where secret information is embedded in the encrypted most significant bit plane. Not only the original image can be recovered with high quality but also the payload can reach close to 1 bit per pixel (bpp). However, there are potential errors in the process of extracting secret data, because most significant bits of a part of pixels are used as flags to mark prediction error location. In this paper, a reversible data hiding method in encrypted image with high capacity is proposed by combining most significant bit prediction with least significant bit compression. At first, most significant bit of each pixel is predicted and a location map of prediction errors in the original image is generated. In the same time, the original image is encrypted using a stream cipher method. Then, the location map is embedded into the vacated space generated with compressing least significant bits and the secret data is embedded into most significant bits of a part of pixels without prediction errors. In this way, the marked encrypted image is obtained. Finally, the original image can be recovered without any error and the secret information can be extracted correctly from the marked encrypted image. Experimental results show that the proposed method has better performance than EPE-based and other methods.