Reviewerly/exHarmony huggingface dataset:
A Unified Method for Deinterleaving and PRI Modulation Recognition of Radar Pulses Based on Deep Neural Networks

In the modern electronic warfare signal environment, multiple radar signals of high density are mixed and received, and separating them into signals for each emitter is an essential step for emitter identification. Each radar has its own pulse repetition interval (PRI), which is a key parameter for deinterleaving pulse trains. The PRI is modulated in various forms depending on the purpose of the radar operation, and analyzing the mean PRI and the modulation type of PRI is the core of electronic warfare signal processing. Many existing papers have tried separate independent approaches for deinterleaving and for PRI modulation recognition. However, many distortions are unintentionally generated in the process of extracting the pulse train using the PRI estimated through deinterleaving for the PRI modulation recognition. This degrades the modulation recognition performance. In this paper, we propose a unified method for the deinterleaving and PRI modulation recognition of radar pulses using deep learning-based multitasking learning. The simulation results demonstrate the good performance of the proposed method for deinterleaving and modulation recognition, compared to the conventional method, and prove that the proposed method is robust in noisy radar signal environments.
Semi-supervised multi-view learning by using label propagation based non-negative matrix factorization

Semi-supervised multi-view learning methods aim to boost the learning performance by conjunction with labeled data, because the label information can enhance the discriminant ability of the learned model. Recently, non-negative matrix factorization has received widespread attention in semi-supervised multi-view learning due to its powerful ability of feature extraction. However, the problem of very limited available labeled data is seldom considered. In this case, it is hard for existing approaches to obtain satisfied performance. Motivated by that the label propagation can classify a large number of unlabeled data with few labeled data, in this paper, we propose a novel semi-supervised multi-view learning approach to address the problem of sparse labeled data, called Label Propagation based Non-negative Matrix Factorization (LPNMF). In our model, the intrinsic manifold structure of data is constructed by the adaptive neighbors technology. Based on this intrinsic manifold structure, the label propagation is further employed to make full use of the limited labeled data. Besides, we design an efficient alternating algorithm for solving the optimization problem and provide theoretical analysis on its convergence properties and computational complexity. Finally, experiments on four real-world datasets demonstrate the advantage of our proposed methods, with comparison to the state-of-the-art methods.
How to Pretrain for Steganalysis

In this paper, we investigate the effect of pretraining CNNs on ImageNet on their performance when refined for steganalysis of digital images. In many cases, it seems that just 'seeing' a large number of images helps with the convergence of the network during the refinement no matter what the pretraining task is. To achieve the best performance, the pretraining task should be related to steganalysis, even if it is done on a completely mismatched cover and stego datasets. Furthermore, the pretraining does not need to be carried out for very long and can be done with limited computational resources. An additional advantage of the pretraining is that it is done on color images and can later be applied for steganalysis of color and grayscale images while still having on-par or better performance than detectors trained specifically for a given source. The refining process is also much faster than training the network from scratch. The most surprising part of the paper is that networks pretrained on JPEG images are a good starting point for spatial domain steganalysis as well.
Manipulation Detection in Satellite Images Using Vision Transformer

A growing number of commercial satellite companies provide easily accessible satellite imagery. Overhead imagery is used by numerous industries including agriculture, forestry, natural disaster analysis, and meteorology. Satellite images, just as any other images, can be tampered with image manipulation tools. Manipulation detection methods created for images captured by "consumer cameras" tend to fail when used on satellite images due to the differences in image sensors, image acquisition, and processing. In this paper we propose an unsupervised technique that uses a Vision Transformer to detect spliced areas within satellite images. We introduce a new dataset which includes manipulated satellite images that contain spliced objects. We show that our proposed approach performs better than existing unsupervised splicing detection techniques.
ReMix: Towards Image-to-Image Translation with Limited Data

Image-to-image (I2I) translation methods based on generative adversarial networks (GANs) typically suffer from overfitting when limited training data is available. In this work, we propose a data augmentation method (ReMix) to tackle this issue. We interpolate training samples at the feature level and propose a novel content loss based on the perceptual relations among samples. The generator learns to translate the in-between samples rather than memorizing the training set, and thereby forces the discriminator to generalize. The proposed approach effectively reduces the ambiguity of generation and renders content-preserving results. The ReMix method can be easily incorporated into existing GAN models with minor modifications. Experimental results on numerous tasks demonstrate that GAN models equipped with the ReMix method achieve significant improvements.
NTIRE 2021 Learning the Super-Resolution Space Challenge

This paper reviews the NTIRE 2021 challenge on learning the super-Resolution space. It focuses on the participating methods and final results. The challenge addresses the problem of learning a model capable of predicting the space of plausible super-resolution (SR) images, from a single low-resolution image. The model must thus be capable of sampling diverse outputs, rather than just generating a single SR image. The goal of the challenge is to spur research into developing learning formulations and models better suited for the highly ill-posed SR problem. And thereby advance the state-of-the-art in the broader SR field. In order to evaluate the quality of the predicted SR space, we propose a new evaluation metric and perform a comprehensive analysis of the participating methods. The challenge contains two tracks: 4× and 8 scale factor. In total, 11 teams competed in the final testing× phase.
On the Improvement of the Isolation Forest Algorithm for Outlier Detection with Streaming Data

In recent years, detecting anomalies in real-world computer networks has become a more and more challenging task due to the steady increase of high-volume, high-speed and high-dimensional streaming data, for which ground truth information is not available. Efficient detection schemes applied on networked embedded devices need to be fast and memory-constrained, and must be capable of dealing with concept drifts when they occur. Different approaches for unsupervised online outlier detection have been designed to deal with these circumstances in order to reliably detect malicious activity. In this paper, we introduce a novel framework called PCB-iForest, which generalized, is able to incorporate any ensemble-based online OD method to function on streaming data. Carefully engineered requirements are compared to the most popular state-of-the-art online methods with an in-depth focus on variants based on the widely accepted isolation forest algorithm, thereby highlighting the lack of a flexible and efficient solution which is satisfied by PCB-iForest. Therefore, we integrate two variants into PCB-iForest—an isolation forest improvement called extended isolation forest and a classic isolation forest variant equipped with the functionality to score features according to their contributions to a sample’s anomalousness. Extensive experiments were performed on 23 different multi-disciplinary and security-related real-world datasets in order to comprehensively evaluate the performance of our implementation compared with off-the-shelf methods. The discussion of results, including AUC, F1 score and averaged execution time metric, shows that PCB-iForest clearly outperformed the state-of-the-art competitors in 61% of cases and even achieved more promising results in terms of the tradeoff between classification and computational costs.
Style-Aware Normalized Loss for Improving Arbitrary Style Transfer

Neural Style Transfer (NST) has quickly evolved from single-style to infinite-style models, also known as Arbitrary Style Transfer (AST). Although appealing results have been widely reported in literature, our empirical studies on four well-known AST approaches (GoogleMagenta [14], AdaIN [19], LinearTransfer [29], and SANet [37]) show that more than 50% of the time, AST stylized images are not acceptable to human users, typically due to under- or over-stylization. We systematically study the cause of this imbalanced style transferability (IST ) and propose a simple yet effective solution to mitigate this issue. Our studies show that the IST issue is related to the conventional AST style loss, and reveal that the root cause is the equal weightage of training samples irrespective of the properties of their corresponding style images, which biases the model towards certain styles. Through investigation of the theoretical bounds of the AST style loss, we propose a new loss that largely overcomes IST . Theoretical analysis and experimental results validate the effectiveness of our loss, with over 80% relative improvement in style deception rate and 98% relatively higher preference in human evaluation.
Triple-cooperative Video Shadow Detection

Shadow detection in a single image has received significant research interests in recent years. However, much fewer works have been explored in shadow detection over dynamic scenes. The bottleneck is the lack of a well-established dataset with high-quality annotations for video shadow detection. In this work, we collect a new video shadow detection dataset (ViSha), which contains 120 videos with 11,685 frames, covering 60 object categories, varying lengths, and different motion/lighting conditions. All the frames are annotated with a high-quality pixel-level shadow mask. To the best of our knowledge, this is the first learning-oriented dataset for video shadow detection. Furthermore, we develop a new baseline model, named triple-cooperative video shadow detection network (TVSD-Net). It utilizes triple parallel networks in a cooperative manner to learn discriminative representations at intra-video and inter-video levels. Within the network, a dual gated co-attention module is proposed to constrain features from neighboring frames in the same video, while an auxiliary similarity loss is introduced to mine semantic information between different videos. Finally, we conduct a comprehensive study on ViSha, evaluating 12 state-of-the-art models (including single image shadow detectors, video object segmentation, and saliency detection methods). Experiments demonstrate that our model outperforms SOTA competitors.
EDPN: Enhanced Deep Pyramid Network for Blurry Image Restoration

Image deblurring has seen a great improvement with the development of deep neural networks. In practice, how-ever, blurry images often suffer from additional degradations such as downscaling and compression. To address these challenges, we propose an Enhanced Deep Pyramid Network (EDPN) for blurry image restoration from multiple degradations, by fully exploiting the self- and cross-scale similarities in the degraded image. Specifically, we design two pyramid-based modules, i.e., the pyramid progressive transfer (PPT) module and the pyramid self-attention (PSA) module, as the main components of the proposed network. By taking several replicated blurry images as inputs, the PPT module transfers both self-and cross-scale similarity information from the same de-graded image in a progressive manner. Then, the PSA module fuses the above transferred features for subsequent restoration using self- and spatial-attention mechanisms. Experimental results demonstrate that our method significantly outperforms existing solutions for blurry image super-resolution and blurry image deblocking. In the NTIRE 2021 Image Deblurring Challenge, EDPN achieves the best PSNR/SSIM/LPIPS scores in Track 1 (Low Resolution) and the best SSIM/LPIPS scores in Track 2 (JPEG Artifacts). The implementation code is available at https://github.com/zeyuxiao1997/EDPN.
A Novel Automatic Modulation Classification Scheme Based on Multi-Scale Networks

Automatic modulation classification enables intelligent communications and it is of crucial importance in today’s and future wireless communication networks. Although many automatic modulation classification schemes have been proposed, they cannot tackle the intra-class diversity problem caused by the dynamic changes of the wireless communication environment. In order to overcome this problem, inspired by face recognition, a novel automatic modulation classification scheme is proposed by using the multi-scale network in this paper. Moreover, a novel loss function that combines the center loss and the cross entropy loss is exploited to learn both discriminative and separable features in order to further improve the classification performance. Extensive simulation results demonstrate that our proposed automatic modulation classification scheme can achieve better performance than the benchmark schemes in terms of the classification accuracy. The influence of the network parameters and the loss function with the two-stage training strategy on the classification accuracy of our proposed scheme are investigated.
Image De-raining via Continual Learning

While deep convolutional neural networks (CNNs) have achieved great success on image de-raining task, most existing methods can only learn fixed mapping rules between paired rainy/clean images on a single dataset. This limits their applications in practical situations with multiple and incremental datasets where the mapping rules may change for different types of rain streaks. However, the catastrophic forgetting of traditional deep CNN model challenges the design of generalized framework for multiple and incremental datasets. A strategy of sharing the network structure but in-dependently updating and storing the network parameters on each dataset has been developed as a potential solution. Nevertheless, this strategy is not applicable to compact systems as it dramatically increases the overall training time and parameter space. To alleviate such limitation, in this study, we propose a parameter importance guided weights modification approach, named PIGWM. Specifically, with new dataset (e.g. new rain dataset), the well-trained network weights are updated according to their importance evaluated on previous training dataset. With extensive experimental validation, we demonstrate that a single network with a single parameter set of our proposed method can process multiple rain datasets almost without performance degradation. The proposed model is capable of achieving superior performance on both inhomogeneous and incremental datasets, and is promising for highly compact systems to gradually learn myriad regularities of the different types of rain streaks. The results indicate that our proposed method has great potential for other computer vision tasks with dynamic learning environments.
Concept Drift Detection from Multi-Class Imbalanced Data Streams

Continual learning from data streams is among the most important topics in contemporary machine learning. One of the biggest challenges in this domain lies in creating algorithms that can continuously adapt to arriving data. However, previously learned knowledge may become outdated, as streams evolve over time. This phenomenon is known as concept drift and must be detected to facilitate efficient adaptation of the learning model. While there exists a plethora of drift detectors, all of them assume that we are dealing with roughly balanced classes. In the case of imbalanced data streams, those detectors will be biased towards the majority classes, ignoring changes happening in the minority ones. Furthermore, class imbalance may evolve over time and classes may change their roles (majority becoming minority and vice versa). This is especially challenging in the multi-class setting, where relationships among classes become complex. In this paper, we propose a detailed taxonomy of challenges posed by concept drift in multi-class imbalanced data streams, as well as a novel trainable concept drift detector based on Restricted Boltzmann Machine. It is capable of monitoring multiple classes at once and using reconstruction error to detect changes in each of them independently. Our detector utilizes a skew-insensitive loss function that allows it to handle multiple imbalanced distributions. Due to its trainable nature, it is capable of following changes in a stream and evolving class roles, as well as it can deal with local concept drift occurring in minority classes. Extensive experimental study on multi-class drifting data streams, enriched with a detailed analysis of the impact of local drifts and changing imbalance ratios, confirms the high efficacy of our approach.
KAZE-SAR: SAR Image Registration Using KAZE Detector and Modified SURF Descriptor for Tackling Speckle Noise

Synthetic aperture radar (SAR) image registration is still a challenging task in remote sensing. The scale-invariant feature transform (SIFT) method and its extensions are most widely used feature detectors in SAR images. However, due to the presence of nonlinear speckle noise, some wrong features are chosen which directly influence the feature matching process. In this article, for the first time, the KAZE algorithm with a modified version of speeded-up robust features (SURF) descriptor is used for SAR image registration. It uses discrete stochastic second order nonlinear partial differential equations (PDEs) to model the SAR image’s edge structure. KAZE uses nonlinear diffusion filtering to build up the scale levels of the SIFT descriptor. It preserves the edges while simultaneously decreasing the speckle noise in smoothing the image. Therefore, it captures the exact location of features in down-sampling and filtering stages of SIFT. Experimental results show that our proposed method has increased the localization accuracy and distinctiveness of feature extraction through the use of KAZE detector. Our method is also able to handle challenging cases in different image sources, different view angles, different image times, complex affine mapping, presence of blurring, and noise.
HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embedding

Scene flow in 3D point clouds plays an important role in understanding dynamic environments. Although significant advances have been made by deep neural networks, the performance is far from satisfactory as only per-point translational motion is considered, neglecting the constraints of the rigid motion in local regions. To address the issue, we propose to introduce the motion consistency to force the smoothness among neighboring points. In addition, constraints on the rigidity of the local transformation are also added by sharing unique rigid motion parameters for all points within each local region. To this end, a high-order CRFs based relation module (Con-HCRFs) is deployed to explore both point-wise smoothness and region-wise rigidity. To empower the CRFs to have a discriminative unary term, we also introduce a position-aware flow estimation module to be incorporated into the Con-HCRFs. Comprehensive experiments on FlyingThings3D and KITTI show that our proposed framework (HCRF-Flow) achieves state-of-the-art performance and significantly outperforms previous approaches substantially.
A Self-boosting Framework for Automated Radiographic Report Generation

Automated radiographic report generation is a challenging task since it requires to generate paragraphs describing fine-grained visual differences of cases, especially for those between the diseased and the healthy. Existing image captioning methods commonly target at generic images, and lack mechanism to meet this requirement. To bridge this gap, in this paper, we propose a self-boosting framework that improves radiographic report generation based on the cooperation of the main task of report generation and an auxiliary task of image-text matching. The two tasks are built as the two branches of a network model and influence each other in a cooperative way. On one hand, the image-text matching branch helps to learn highly text-correlated visual features for the report generation branch to output high quality reports. On the other hand, the improved reports produced by the report generation branch provide additional harder samples for the image-text matching branch and enforce the latter to improve itself by learning better visual and text feature representations. This, in turn, helps improve the report generation branch again. These two branches are jointly trained to help improve each other iteratively and progressively, so that the whole model is self-boosted without requiring external resources. Experimental results demonstrate the effectiveness of our method on two public datasets, showing its superior performance over multiple state-of-the-art image captioning and medical report generation methods.
A parallel learning particle swarm optimizer for inverse kinematics of robotic manipulator

In this study, a novel parallel learning particle swarm optimizer (PLPSO) is proposed. The evolutionary strategy of the algorithm is quite different from that of the existing PSO algorithms. To enhance the global search capability of the particle swarm, the original particle swarm is divided into two parallel evolving independent particle subpopulations to explore the search space simultaneously. In addition, according to the evolutionary factor in each iteration, the poorly performing particles in the two subpopulations are spurred and learned to improve the search speed of the particle swarm. In the process of particle learning, delay information is added to the velocity update item to reduce the occurrence of local capture. Twenty-eight benchmark functions of CEC2013 are used to evaluate the performance of the proposed PLPSO algorithm. Numerous comparisons demonstrate that the performance of the PLPSO algorithm is better than that of other nine PSO algorithms. Afterwards, the PLPSO algorithm is applied to the inverse kinematics (IK) solution of a robotic manipulator. By comparing with other three intelligent algorithms, the PLPSO algorithm shows an excellent performance in solving the IK problem. Finally, tests are carried out on an UR5 manipulator to confirm the practicability of the proposed PLPSO algorithm further. Thus, the feasibility of using the PLPSO algorithm for solving the IK problem of a robotic manipulator is verified.