Reviewerly/exHarmony huggingface dataset:
Image steganography based on style transfer and quaternion exponent moments

Although the existing steganography methods can successfully embed secret information into the carrier image without introducing distortion into the appearance of the carrier image, the difference in distribution between the carrier image and stego image still cannot resist the detection by statistics-based steganalysis algorithms. To improve the capability of resisting steganalysis algorithms, an image steganography scheme based on style transfer and quaternion exponent moments is proposed in this paper. First, the geometric invariance of quaternion exponent moments is combined to accomplish the task of embedding secret information. Next, the style transfer is performed on a stego image embedded with the secret information, and the stylized image is used to transmit through the common channel. Then, the receiver attempts to remove the style from the stylized image and to restore the stylized image back to its original appearance. For this purpose, a de-stylized network is designed to reconstruct the stego image from the stylized image. Finally, an extracting algorithm is used to extract the transmitted secret image from the reconstructed stego image. In the steganography process, the main goal is to achieve that even when the appearance and distribution of the carrier image have been changed, that should also look like an independent and normal behavior to an eavesdropper. Extensive experiments are conducted to verify the feasibility of the proposed scheme. Experimental and analysis results indicate that the proposed scheme can generate an independent and meaningful image and successfully transmit a secret image and has the ability to extract a secret image at a low bit error rate. In addition, the proposed scheme provides high security.
Automatic Modulation Recognition Based on Adaptive Attention Mechanism and ResNeXt WSL Model

Automatic modulation recognition (AMR) plays an important role in modern wireless communication. In this letter, a novel framework for AMR is proposed. The ResNeXt network serves as the backbone, and four proposed adaptive attention mechanism modules are incorporated. The time-frequency representations of the received signals are utilized as the inputs of the proposed deep learning (DL) network, and a transfer learning strategy is adopted based on the pre-trained ResNeXt weakly supervised learning (WSL) model. The comparisons with several state-of-the-art techniques on the RadioML2016.10B and RadioML2018.01A datasets show that the proposed framework converges quickly and can achieve higher robustness and 2% to 8% higher accuracy than other state-of-the-art techniques.
Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation

Action segmentation refers to inferring boundaries of semantically consistent visual concepts in videos and is an important requirement for many video understanding tasks. For this and other video understanding tasks, supervised approaches have achieved encouraging performance but require a high volume of detailed frame-level annotations. We present a fully automatic and unsupervised approach for segmenting actions in a video that does not require any training. Our proposal is an effective temporally-weighted hierarchical clustering algorithm that can group semantically consistent frames of the video. Our main finding is that representing a video with a 1-nearest neighbor graph by taking into account the time progression is sufficient to form semantically and temporally consistent clusters of frames where each cluster may represent some action in the video. Additionally, we establish strong unsupervised baselines for action segmentation and show significant performance improvements over published unsupervised methods on five challenging action segmentation datasets. Our code is available. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation

Domain adaptation aims to learn a transferable model to bridge the domain shift between one labeled source domain and another sparsely labeled or unlabeled target domain. Since the labeled data may be collected from multiple sources, multi-source domain adaptation (MDA) has attracted increasing attention. Recent MDA methods do not consider the pixel-level alignment between sources and target or the misalignment across different sources. In this paper, we propose a novel MDA framework to address these challenges. Specifically, we design a novel Multi-source Adversarial Domain Aggregation Network (MADAN). First, an adapted domain is generated for each source with dynamic semantic consistency while aligning towards the target at the pixel-level cycle-consistently. Second, sub-domain aggregation discriminator and cross-domain cycle discriminator are proposed to make different adapted domains more closely aggregated. Finally, feature-level alignment is performed between the aggregated domain and the target domain while training the task network. For the segmentation adaptation, we further enforce category-level alignment and incorporate multi-scale image generation, which constitutes MADAN+. We conduct extensive MDA experiments on digit recognition, object classification, and simulation-to-real semantic segmentation tasks. The results demonstrate that the proposed MADAN and MADAN+ models outperform state-of-the-art approaches by a large margin.
Stock exchange trading optimization algorithm: a human-inspired method for global optimization

In this paper, a human-inspired optimization algorithm called stock exchange trading optimization (SETO) for solving numerical and engineering problems is introduced. The inspiration source of this optimizer is the behavior of traders and stock price changes in the stock market. Traders use various fundamental and technical analysis methods to gain maximum profit. SETO mathematically models the technical trading strategy of traders to perform optimization. It contains three main actuators including rising, falling, and exchange. These operators navigate the search agents toward the global optimum. The proposed algorithm is compared with seven popular meta-heuristic optimizers on forty single-objective unconstraint numerical functions and four engineering design problems. The statistical results obtained on test problems show that SETO is capable of providing competitive and promising performances compared with counterpart algorithms in solving optimization problems of different dimensions, especially 1000-dimension problems. Out of 40 numerical functions, the SETO algorithm has achieved the global optimum on 36 functions, and out of 4 engineering problems, it has obtained the best results on 3 problems.
Rich Context Aggregation with Reflection Prior for Glass Surface Detection

Glass surfaces appear everywhere. Their existence can however pose a serious problem to computer vision tasks. Recently, a method is proposed to detect glass surfaces by learning multi-scale contextual information. However, as it is only based on a general context integration operation and does not consider any specific glass surface properties, it gets confused when the images contain objects that are similar to glass surfaces and degenerates in challenging scenes with insufficient contexts. We observe that humans often rely on identifying reflections in order to sense the existence of glass and on locating the boundary in order to deter-mine the extent of the glass. Hence, we propose a model for glass surface detection, which consists of two novel modules: (1) a rich context aggregation module (RCAM) to ex-tract multi-scale boundary features from rich context features for locating glass surface boundaries of different sizes and shapes, and (2) a reflection-based refinement module (RRM) to detect reflection and then incorporate it so as to differentiate glass regions from non-glass regions. In addition, we also propose a challenging dataset consisting of 4,012 glass images with annotations for glass surface detection. Our experiments demonstrate that the proposed model outperforms state-of-the-art methods from relevant fields.
An Occlusion-aware Multi-target Multi-camera Tracking System

Multi-camera tracking of vehicles on a city-scale level is a crucial task for efficient traffic monitoring. Most of the errors made by such multi-target multi-camera tracking systems arise due to tracking failures or misleading visual information of detection boxes under occlusion. Therefore, we propose an occlusion-aware approach that leverages temporal information from tracks to improve the single-camera tracking performance by an occlusion handling strategy and additional modules to filter false detections. For the multi-camera tracking, we discard obstacle-occluded detection boxes by a background filtering technique and boxes overlapping with other targets using the available track information to improve the quality of extracted visual features. Furthermore, topological and temporal constraints are incorporated to simplify the re-identification task in the multi-camera clustering. We give detailed insights into our method with ablative experiments and show its competitiveness on the CityFlowV2 dataset, where we achieve promising results ranking 4th in Track 3 of the 2021 AI City Challenge.
Dual Attention Transfer in Session-based Recommendation with Multi-dimensional Integration

Session-based recommendation (SBR) is widely used in e-commerce to predict the anonymous user's next click action according to a short sequence. Many previous studies have shown the potential advantages of applying Graph Neural Networks (GNN) to SBR tasks. However, the existing SBR models using GNN to solve user preference problems are only based on one single dataset to obtain one recommendation model during training. While the single dataset has the problems including the excessive sparse data source and the long-distance relationship of items. Therefore, introducing the dual transfer, which can enrich the data source, to SBR is absolutely necessary. To this end, a new method is proposed in this paper, which is called dual attention transfer based on multi-dimensional integration (DAT-MDI): (i) DAT uses a potential mapping method based on a slot attention mechanism to extract the user's representation information in different sessions between multiple domains. (ii) MDI combines the graph neural network for the graphs (session graph and global graph) and the gate recurrent unit (GRU) for the sequence to learn the item representation in each session. Then the multi-level session representation are combined by a soft-attention mechanism. We do a variety of experiments on four benchmark datasets which have shown that the superiority of the DAT-MDI model over the state-of-the-art methods.
Underwater image restoration via depth map and illumination estimation based on a single image

For the enhancement process of underwater images taken in various water types, previous methods employ the simple image formation model, thus obtaining poor restoration results. Recently, a revised underwater image formation model (i.e., the Akkaynak-Treibitz model) has shown better robustness in underwater image restoration, but has drawn little attention due to its complexity. Herein, we develop a dehazing method utilizing the revised model, which depends on the scene depth map and a color correction method to eliminate color distortion. Specifically, we first design an underwater image depth estimation method to create the depth map. Subsequently, according to the depth value of each pixel, the backscatter is estimated and removed by the channel based on the revised model. Furthermore, we propose a color correction approach to adjust the global color distribution of the image automatically. Our method only uses a single underwater image as input to eliminate lightwave absorption and scattering influence. Compared with state-of-the-art methods, both subjective and objective experimental results show that our approach can be applied to various real-world underwater scenes and has better contrast and color.
Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning Over Noisy Channels

We propose a novel formulation of the “effectiveness problem” in communications, put forth by Shannon and Weaver in their seminal work “The Mathematical Theory of Communication”, by considering multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation in a multi-agent reinforcement learning (MARL) framework. Specifically, we consider a multi-agent partially observable Markov decision process (MA-POMDP), in which the agents, in addition to interacting with the environment, can also communicate with each other over a noisy communication channel. The noisy communication channel is considered explicitly as part of the dynamics of the environment, and the message each agent sends is part of the action that the agent can take. As a result, the agents learn not only to collaborate with each other but also to communicate “effectively” over a noisy channel. This framework generalizes both the traditional communication problem, where the main goal is to convey a message reliably over a noisy channel, and the “learning to communicate” framework that has received recent attention in the MARL literature, where the underlying communication channels are assumed to be error-free. We show via examples that the joint policy learned using the proposed framework is superior to that where the communication is considered separately from the underlying MA-POMDP. This is a very powerful framework, which has many real world applications, from autonomous vehicle planning to drone swarm control, and opens up the rich toolbox of deep reinforcement learning for the design of multi-user communication systems.
ASSD: Feature Aligned Single-Shot Detection for Multiscale Objects in Aerial Imagery

Object detection is a fundamental part of the interpretation of remote sensing imagery. The one-stage object detector has been adopted into this field because of its high computational efficiency. However, this detector suffers from the misalignment among predefined anchor, object, and feature extracted by standard convolution kernel both in spatial and scale. It limits the further improvement of performance, especially for the long-narrow and multiscale geospatial objects. In this article, the problem is defined as <i>the feature misalignment</i> problem. To deal with this issue, an efficient feature aligned single-shot detector (ASSD) is proposed, which consists of two modules: a novel pseudo anchor proposal module (PAPM) and a flexible context-based feature alignment module (CFAM). The PAPM replaces the regular anchor group with the proposed core anchor and refines it to get aligned locations. It can tackle the spatial misalignment between anchors and their corresponding objects and alleviate the negative/positive imbalance problem. Then, the CFAM adaptively adjusts the sampling points of the convolution kernel and collects the context information according to the aligned core anchor. This plug-and-play module can effectively rectify the misalignment between kernel and objects and extract aligned and robust features. A series of comprehensive experiments are conducted on two large-scale public remote sensing object detection datasets. Experiment results suggest that the proposed method is effective to alleviate the misalignment problem. Compared with the baseline model, the detection accuracy is improved by 8.5&#x0025; mAP and 11.0&#x0025; mAP on the challenging benchmark for object detection in optical remote sensing image (DIOR) and a large-scale dataset for object detection in aerial image (DOTA) dataset, respectively. Our best-resulting model achieves the state-of-the-art performance, surpassing other one-stage detectors both on the two datasets at a high detection speed of 21 FPS.
Online reinforcement learning for a continuous space system with experimental validation

Reinforcement learning (RL) for continuous state/action space systems has remained a challenge for nonlinear multivariate dynamical systems even at a simulation level. Implementing such schemes for real-time control is still of a difficulty and remains largely unanswered. In this study, several critical strategies for practical implementation of RL are developed, and a multivariable, multi-modal, hybrid three-tank (HTT) physical process is utilized to illustrate the proposed strategies. A successful real-time implementation of RL is reported. The first step is a meta-heuristic first principles model parameter optimization, where a custom pseudo random binary signal (PRBS) is used to obtain open-loop experimental data. This is followed by in silico asynchronous advantage actor–critic (A3C/A-A2C) based policy learning. In the second step, three different approaches (namely proximal learning, single trajectory learning, and multiple trajectory learning) are utilized to explore the state/action space. In the final step, online learning (A2C) using the best in silico policy on the real process using a socket connection is established. The extent of exploration (EoE, a measure of exploration) is proposed as a parameter for quantifying exploration of the state/action space. While the online sample efficiency of RL application is enhanced, a soft constraint based constrained learning is proposed and validated. With considerations of the proposed strategies, this work demonstrates the possibility of applying RL to solve practical control problems.
Information Bottleneck Disentanglement for Identity Swapping

Improving the performance of face forgery detectors often requires more identity-swapped images of higher-quality. One core objective of identity swapping is to generate identity-discriminative faces that are distinct from the target while identical to the source. To this end, properly disentangling identity and identity-irrelevant information is critical and remains a challenging endeavor. In this work, we propose a novel information disentangling and swapping network, called InfoSwap, to extract the most expressive information for identity representation from a pre-trained face recognition model. The key insight of our method is to formulate the learning of disentangled representations as optimizing an information bottleneck tradeoff, in terms of finding an optimal compression of the pretrained latent features. Moreover, a novel identity contrastive loss is proposed for further disentanglement by requiring a proper distance between the generated identity and the target. While the most prior works have focused on using various loss functions to implicitly guide the learning of representations, we demonstrate that our model can provide explicit supervision for learning disentangled representations, achieving impressive performance in generating more identity-discriminative swapped faces.
DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

Binary grid mask representation is broadly used in instance segmentation. A representative instantiation is Mask R-CNN which predicts masks on a 28×28 binary grid. Generally, a low-resolution grid is not sufficient to capture the details, while a high-resolution grid dramatically increases the training complexity. In this paper, we propose a new mask representation by applying the discrete cosine transform(DCT) to encode the high-resolution binary grid mask into a compact vector. Our method, termed DCT-Mask, could be easily integrated into most pixel-based instance segmentation methods. Without any bells and whistles, DCT-Mask yields significant gains on different frameworks, backbones, datasets, and training schedules. It does not require any pre-processing or pre-training, and almost no harm to the running speed. Especially, for higher-quality annotations and more complex backbones, our method has a greater improvement. Moreover, we analyze the performance of our method from the perspective of the quality of mask representation. The main reason why DCT-Mask works well is that it obtains a high-quality mask representation with low complexity.
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity – model performances are largely bottlenecked by the lack of sketch-photo pairs. Whilst the number of photos can be easily scaled, each corresponding sketch still needs to be individually produced. In this paper, we aim to mitigate such an upper-bound on sketch data, and study whether unlabelled photos alone (of which they are many) can be cultivated for performance gain. In particular, we introduce a novel semi-supervised framework for cross-modal retrieval that can additionally leverage large-scale unlabelled photos to account for data scarcity. At the center of our semi-supervision design is a sequential photo-to-sketch generation model that aims to generate paired sketches for unlabelled photos. Importantly, we further introduce a discriminator-guided mechanism to guide against unfaithful generation, together with a distillation loss-based regularizer to provide tolerance against noisy training samples. Last but not least, we treat generation and retrieval as two conjugate problems, where a joint learning procedure is devised for each module to mutually benefit from each other. Extensive experiments show that our semi-supervised model yields a significant performance boost over the state-of-the-art supervised alternatives, as well as existing methods that can exploit unlabelled photos for FG-SBIR.
Generalizing state-of-the-art object detectors for autonomous vehicles in unseen environments

In scene understanding for autonomous vehicles (AVs), models trained on the available datasets fail to generalize well to the complex, real-world scenarios with higher dynamics. In this work, we attempt to handle the distribution mismatch by employing the generative adversarial network (GAN) and weather modeling to strengthen the intra-domain data. We also alleviate the fragility of our trained models against natural distortions with state-of-the-art augmentation approaches. Finally, we assess our method for cross-domain object detection through CARLA simulation. Our experiments demonstrate that: (1) Augmenting training class with even limited intra-domain data captured from the adverse weather conditions boosts the generalization of the two kinds of object detectors; (2) Exploiting GANs and weather modeling to elaborately simulate the adverse, intra-domain weather conditions manages to surmount the adverse data scarcity issue for intra-domain object detection; (3) A combination of Augmix and style augmentations not only can promote the robustness of our trained models against different natural distortions but also can boost their performance in the cross-domain object detection; (4) Training GANs for unsupervised image-to-image translation by means of the existing, large-scale datasets outside of our training domain is found beneficial to alleviate image-based and instance-based domain shifts.