Reviewerly/exHarmony huggingface dataset:
Quantum-enhanced multiobjective large-scale optimization via parallelism

Traditional quantum-based evolutionary algorithms are intended to solve single-objective optimization problems or multiobjective small-scale optimization problems. However, multiobjective large-scale optimization problems are continuously emerging in the big-data era. Therefore, the research in this paper, which focuses on combining quantum mechanics with multiobjective large-scale optimization algorithms, will be beneficial to the study of quantum-based evolutionary algorithms. In traditional quantum-behaved particle swarm optimization (QPSO), particle position uncertainty prevents the algorithm from easily falling into local optima. Inspired by the uncertainty principle of position, the authors propose quantum-enhanced multiobjective large-scale algorithms, which are parallel multiobjective large-scale evolutionary algorithms (PMLEAs). Specifically, PMLEA-QDE, PMLEA-QjDE and PMLEA-QJADE are proposed by introducing the search mechanism of the individual particle from QPSO into differential evolution (DE), differential evolution with self-adapting control parameters (jDE) and adaptive differential evolution with optional external archive (JADE). Moreover, the proposed algorithms are implemented with parallelism to improve the optimization efficiency. Verifications performed on several test suites indicate that the proposed quantum-enhanced algorithms are superior to the state-of-the-art algorithms in terms of both effectiveness and efficiency.
Deep Face Super-Resolution With Iterative Collaboration Between Attentive Recovery and Landmark Estimation

Recent works based on deep learning and facial priors have succeeded in super-resolving severely degraded facial images. However, the prior knowledge is not fully exploited in existing methods, since facial priors such as landmark and component maps are always estimated by low-resolution or coarsely super-resolved images, which may be inaccurate and thus affect the recovery performance. In this paper, we propose a deep face super-resolution (FSR) method with iterative collaboration between two recurrent networks which focus on facial image recovery and landmark estimation respectively. In each recurrent step, the recovery branch utilizes the prior knowledge of landmarks to yield higher-quality images which facilitate more accurate landmark estimation in turn. Therefore, the iterative information interaction between two processes boosts the performance of each other progressively. Moreover, a new attentive fusion module is designed to strengthen the guidance of landmark maps, where facial components are generated individually and aggregated attentively for better restoration. Quantitative and qualitative experimental results show the proposed method significantly outperforms state-of-the-art FSR methods in recovering high-quality face images.
UniPose: Unified Human Pose Estimation in Single Images and Videos

We propose UniPose, a unified framework for human pose estimation, based on our “Waterfall” Atrous Spatial Pooling architecture, that achieves state-of-art-results on several pose estimation metrics. UniPose incorporates contextual segmentation and joint localization to estimate the human pose in a single stage, with high accuracy, without relying on statistical postprocessing methods. The Waterfall module in UniPose leverages the efficiency of progressive filtering in the cascade architecture, while maintaining multi-scale fields-of-view comparable to spatial pyramid configurations. Additionally, our method is extended to UniPose-LSTM for multi-frame processing and achieves state-of-the-art results for temporal pose estimation in Video. Our results on multiple datasets demonstrate that UniPose, with a ResNet backbone and Waterfall module, is a robust and efficient architecture for pose estimation obtaining state-of-the-art results in single person pose detection for both single images and videos.
An effective image encryption algorithm based on compressive sensing and 2D-SLIM

This paper presents an image encryption algorithm based on the compressive sensing and a hyperchaotic map, which includes the permutation, compression and diffusion processes. Firstly, the plain image is transformed using discrete wavelet transform, and the coefficient matrix is obtained. Then, the row and column permutation are applied to achieve a good scrambling effect. Secondly, the permuted coefficient matrix is measured by a measurement matrix, and the measurement matrix is constructed using chaotic sequences generated by the 2D-SLIM map. Finally, the diffusion algorithm based on Galois field multiplication is designed to encrypt the compressed matrix, and the cipher image with small size and high security is obtained. What’s more, the SHA-512 hash value of the plain image is introduced to set the initial values and parameters of the 2D-SLIM map to establish the correlation between the plain image and the algorithm, and it makes the algorithm can resist the known/chosen plaintext attacks. Furthermore, the row and column permutation before compressive sensing successfully improves the compression effect with low compression radio, and the diffusion operation improves the security of this algorithm. Simulations and performance analyses validate the good compression performance and high security of the proposed algorithm.
HyperLi-Net: A hyper-light deep learning network for high-accurate and high-speed ship detection from synthetic aperture radar imagery

Ship detection from Synthetic Aperture Radar (SAR) imagery is attracting increasing attention due to its great value in ocean. However, existing most studies are frequently improving detection accuracy at the expense of detection speed. Thus, to solve this problem, this paper proposes HyperLi-Net for high-accurate and high-speed SAR ship detection. We propose five external modules to achieve high-accuracy, i.e., Multi-Receptive-Field Module (MRF-Module), Dilated Convolution Module (DC-Module), Channel and Spatial Attention Module (CSA-Module), Feature Fusion Module (FF-Module) and Feature Pyramid Module (FP-Module). We also adopt five internal mechanisms to achieve high-speed, i.e., Region-Free Model (RF-Model), Small Kernel (S-Kernel), Narrow Channel (N-Channel), Separable Convolution (Separa-Conv) and Batch Normalization Fusion (BN-Fusion). Experimental results on the SAR Ship Detection Dataset (SSDD), Gaofen-SSDD and Sentinel-SSDD show that HyperLi-Net’s accuracy and speed are both superior to the other nine state-of-the-art methods. Moreover, the satisfactory detection results on two Sentinel-1 SAR images can reveal HyperLi-Net’s good migration capability. HyperLi-Net is build from scratch with fewer parameters, lower computation costs and lighter model that can be efficiently trained on CPUs and is helpful for future hardware transplantation, e.g. FPGAs, DSPs, etc.
Explaining the black-box model: A survey of local interpretation methods for deep neural networks

Recently, a significant amount of research has been investigated on interpretation of deep neural networks (DNNs) which are normally processed as black box models. Among the methods that have been developed, local interpretation methods stand out which have the features of clear expression in interpretation and low computation complexity. Different from existing surveys which cover a broad range of methods on interpretation of DNNs, this survey focuses on local interpretation methods with an in-depth analysis of the representative works including the newly proposed approaches. From the perspective of principles, we first divide local interpretation methods into two main categories: model-driven methods and data-driven methods. Then we make a fine-grained distinction between the two types of these methods, and highlight the latest ideas and principles. We further demonstrate the effects of a number of interpretation methods by reproducing the results through open source software plugins. Finally, we point out research directions in this rapidly evolving field.
RepChain: A Reputation-Based Secure, Fast, and High Incentive Blockchain System via Sharding

In today's blockchain system, designing a secure and high throughput blockchain on par with a centralized payment system is a difficult task. Sharding is one of the most worthwhile emerging technologies for improving the system throughput while maintain high-security level. However, previous sharding-related designs have two main limitations. First, the security and throughput of their random-based sharding system are not high enough as they did not leverage the heterogeneity among validators. Second, to design an incentive mechanism that promotes cooperation could incur a huge overhead on their system. In this article, we propose RepChain, a reputation-based secure and fast blockchain system via sharding, which also provides high incentive to stimulate node cooperation. RepChain utilizes reputation to explicitly characterize the heterogeneity among the validators and lay the foundation for the incentive mechanism. We propose a new double-chain architecture-a transaction chain and a reputation chain. For the transaction chain, an efficient Raft-based synchronous consensus has been presented. For the reputation chain, the synchronous Byzantine fault tolerance consensus that combines collective signing has been utilized to prevent the attack on both reputation score and the related transaction blocks. It supports a high throughput transaction chain with moderate generation speed. Moreover, we propose a reputation-based sharding and leader selection scheme. To analyze the security of RepChain, we propose a recursive formula to calculate the epoch security within only $\mathcal {O}(km^{2})$ time. Furthermore, we implement and evaluate RepChain on the Amazon Web Service platform. The results show our solution can enhance both throughout and security level of the existing sharding-based blockchain system.
Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks

Deep learning-based methods have recently demonstrated promising results in deformable image registration for a wide range of medical image analysis tasks. However, existing deep learning-based methods are usually limited to small deformation settings, and desirable properties of the transformation including bijective mapping and topology preservation are often being ignored by these approaches. In this paper, we propose a deep Laplacian Pyramid Image Registration Network, which can solve the image registration optimization problem in a coarse-to-fine fashion within the space of diffeomorphic maps. Extensive quantitative and qualitative evaluations on two MR brain scan datasets show that our method outperforms the existing methods by a significant margin while maintaining desirable diffeomorphic properties and promising registration speed.
Collaborative Video Object Segmentation by Foreground-Background Integration

This paper investigates the principles of embedding learning to tackle the challenging semi-supervised video object segmentation. Different from previous practices that only explore the embedding learning using pixels from foreground object (s), we consider background should be equally treated and thus propose Collaborative video object segmentation by Foreground-Background Integration (CFBI) approach. Our CFBI implicitly imposes the feature embedding from the target foreground object and its corresponding background to be contrastive, promoting the segmentation results accordingly. With the feature embedding from both foreground and background, our CFBI performs the matching process between the reference and the predicted sequence from both pixel and instance levels, making the CFBI be robust to various object scales. We conduct extensive experiments on three popular benchmarks, i.e., DAVIS 2016, DAVIS 2017, and YouTube-VOS. Our CFBI achieves the performance ( $$\mathcal {J}$$ & $$\mathcal {F}$$ ) of 89.4%, 81.9%, and 81.4%, respectively, outperforming all the other state-of-the-art methods. Code: https://github.com/z-x-yang/CFBI .
Rethinking the Distribution Gap of Person Re-identification with Camera-Based Batch Normalization

The fundamental difficulty in person re-identification (ReID) lies in learning the correspondence among individual cameras. It strongly demands costly inter-camera annotations, yet the trained models are not guaranteed to transfer well to previously unseen cameras. These problems significantly limit the application of ReID. This paper rethinks the working mechanism of conventional ReID approaches and puts forward a new solution. With an effective operator named Camera-based Batch Normalization (CBN), we force the image data of all cameras to fall onto the same subspace, so that the distribution gap between any camera pair is largely shrunk. This alignment brings two benefits. First, the trained model enjoys better abilities to generalize across scenarios with unseen cameras as well as transfer across multiple training sets. Second, we can rely on intra-camera annotations, which have been undervalued before due to the lack of cross-camera information, to achieve competitive ReID performance. Experiments on a wide range of ReID tasks demonstrate the effectiveness of our approach. The code is available at https://github.com/automan000/Camera-based-Person-ReID.
SegFix: Model-Agnostic Boundary Refinement for Segmentation

We present a model-agnostic post-processing scheme to improve the boundary quality for the segmentation result that is generated by any existing segmentation model. Motivated by the empirical observation that the label predictions of interior pixels are more reliable, we propose to replace the originally unreliable predictions of boundary pixels by the predictions of interior pixels. Our approach processes only the input image through two steps: (i) localize the boundary pixels and (ii) identify the corresponding interior pixel for each boundary pixel. We build the correspondence by learning a direction away from the boundary pixel to an interior pixel. Our method requires no prior information of the segmentation models and achieves nearly real-time speed. We empirically verify that our SegFix consistently reduces the boundary errors for segmentation results generated from various state-of-the-art models on Cityscapes, ADE20K and GTA5. Code is available at: https://github.com/openseg-group/openseg.pytorch.
A bibliometric analysis and visualization of blockchain

Blockchain has received global concern as a research focus in recent years. It is a decentralized and unreliably distributed database technology. Numerous countries and researchers have successively applied blockchain in various fields. CiteSpace and VOSviewer were adopted as bibliometric analysis tools in this paper to obtain a comprehensive overview of blockchain technology and grasp the current research emphasis. The Web of Science core collection database was used as the primary source for data collection. The data were collected from 3826 articles published from 2013 to 2020. Specifically, the publication trends and categories might throw light on the development trends and current domains in terms of blockchain research. The most influential and productive authors, institutions, countries, and journals were involved in this bibliometric analysis. Theoretical knowledge and hot research topics on blockchain mainly ranged over “smart contract”, “bitcoin”, “security”, “ethereum”, and “cryptography”. A cluster analysis from the keyword perspective was carried out to obtain emerging trends and frontiers of blockchain. The results showed that future research should concentrate on management, blockchain technology, energy, machine learning, and smart home. Therefore, this paper assisted scholars and practitioners in getting a comprehensive understanding of the status quo and trends of blockchain research.
Matrix Optimization on Universal Unitary Photonic Devices

Universal unitary photonic devices can apply arbitrary unitary transformations to a vector of input modes and provide a promising hardware platform for fast and energy-efficient machine learning using light. We simulate the gradient-based optimization of random unitary matrices on universal photonic devices composed of imperfect tunable interferometers. If device components are initialized uniform-randomly, the locally-interacting nature of the mesh components biases the optimization search space towards banded unitary matrices, limiting convergence to random unitary matrices. We detail a procedure for initializing the device by sampling from the distribution of random unitary matrices and show that this greatly improves convergence speed. We also explore mesh architecture improvements such as adding extra tunable beamsplitters or permuting waveguide layers to further improve the training speed and scalability of these devices.
REMIND Your Neural Network to Prevent Catastrophic Forgetting

People learn throughout life. However, incrementally updating conventional neural networks leads to catastrophic forgetting. A common remedy is replay, which is inspired by how the brain consolidates memory. Replay involves fine-tuning a network on a mixture of new and old instances. While there is neuroscientific evidence that the brain replays compressed memories, existing methods for convolutional networks replay raw images. Here, we propose REMIND, a brain-inspired approach that enables efficient replay with compressed representations. REMIND is trained in an online manner, meaning it learns one example at a time, which is closer to how humans learn. Under the same constraints, REMIND outperforms other methods for incremental class learning on the ImageNet ILSVRC-2012 dataset. We probe REMIND’s robustness to data ordering schemes known to induce catastrophic forgetting. We demonstrate REMIND’s generality by pioneering online learning for Visual Question Answering (VQA) ( https://github.com/tyler-hayes/REMIND ).
Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations

Deep encoder-decoder based CNNs have advanced image inpainting methods for hole filling. While existing methods recover structures and textures step-by-step in the hole regions, they typically use two encoder-decoders for separate recovery. The CNN features of each encoder are learned to capture either missing structures or textures without considering them as a whole. The insufficient utilization of these encoder features hampers the performance of recovering both structures and textures. In this paper, we propose a mutual encoder-decoder CNN for joint recovery of both. We use CNN features from the deep and shallow layers of the encoder to represent structures and textures of an input image, respectively. The deep layer features are sent to a structure branch, while the shallow layer features are sent to a texture branch. In each branch, we fill holes in multiple scales of the CNN features. The filled CNN features from both branches are concatenated and then equalized. During feature equalization, we reweigh channel attentions first and propose a bilateral propagation activation function to enable spatial equalization. To this end, the filled CNN features of structure and texture mutually benefit each other to represent image content at all feature levels. We then use the equalized feature to supplement decoder features for output image generation through skip connections. Experiments on benchmark datasets show that the proposed method is effective to recover structures and textures and performs favorably against state-of-the-art approaches.
Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark

Since real-world objects and their interactions are often multi-modal and multi-typed, heterogeneous networks have been widely used as a more powerful, realistic, and generic superclass of traditional homogeneous networks (graphs). Meanwhile, representation learning (a.k.a. embedding) has recently been intensively studied and shown effective for various network mining and analytical tasks. In this work, we aim to provide a unified framework to deeply summarize and evaluate existing research on heterogeneous network embedding (HNE), which includes but goes beyond a normal survey. Since there has already been a broad body of HNE algorithms, as the first contribution of this work, we provide a generic paradigm for the systematic categorization and analysis over the merits of various existing HNE algorithms. Moreover, existing HNE algorithms, though mostly claimed generic, are often evaluated on different datasets. Understandable due to the application favor of HNE, such indirect comparisons largely hinder the proper attribution of improved task performance towards effective data preprocessing and novel technical design, especially considering the various ways possible to construct a heterogeneous network from real-world application data. Therefore, as the second contribution, we create four benchmark datasets with various properties regarding scale, structure, attribute/label availability, and etc. from different sources, towards handy and fair evaluations of HNE algorithms. As the third contribution, we carefully refactor and amend the implementations and create friendly interfaces for 13 popular HNE algorithms, and provide all-around comparisons among them over multiple tasks and experimental settings. By putting all existing HNE algorithms under a unified framework, we aim to provide a universal reference and guideline for the understanding and development of HNE algorithms. Meanwhile, by open-sourcing all data and code, we envision to serve the community with an ready-to-use benchmark platform to test and compare the performance of existing and future HNE algorithms (https://github.com/yangji9181/HNE).
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE

We tackle the problem of action-conditioned generation of realistic and diverse human motion sequences. In contrast to methods that complete, or extend, motion sequences, this task does not require an initial pose or sequence. Here we learn an action-aware latent representation for human motions by training a generative variational autoencoder (VAE). By sampling from this latent space and querying a certain duration through a series of positional encodings, we synthesize variable-length motion sequences conditioned on a categorical action. Specifically, we design a Transformer-based architecture, ACTOR, for encoding and decoding a sequence of parametric SMPL human body models estimated from action recognition datasets. We evaluate our approach on the NTU RGB+D, HumanAct12 and UESTC datasets and show improvements over the state of the art. Furthermore, we present two use cases: improving action recognition through adding our synthesized data to training, and motion denoising. Code and models are available on our project page.