Reviewerly/exHarmony huggingface dataset:
A new selection operator for differential evolution algorithm

Most research on improving differential evolution algorithms has focused on mutation operator and parameter control. In this paper, a new selection operator is proposed to improve differential evolution algorithm performance. When the individual is not in a state of stagnation, the proposed selection operator is the same as the classical selection operator, meaning that it chooses the best vector from the trial vector and parent vector to survive to the next generation. When the individual is in a state of stagnation, the three other candidate vectors may survive to the next generation. The first candidate vector is the best vector of all the discarded trial vectors of the parent vector. The second candidate vector is the second-best vector of all the discarded trial vectors of the parent vector. The third candidate vector is randomly chosen from all the successfully updated solutions. The proposed selection operator will improve the differential evolution algorithm's ability to escape the local optimal value. 58 benchmark functions are used for verification of the proposed selection operator's performance. Experiments were conducted in order to compare six differential evolution algorithms' performances using the proposed selection operator and not using the proposed selection operator. Simulation results showed that the proposed selection operator significantly improved the differential evolution algorithm's performance.
Cyber sickness in low-immersive, semi-immersive, and fully immersive virtual reality

It is known that virtual reality (VR) experience may cause cyber sickness. One aspect of VR is an immersion or otherwise sense of presence, the sense of feeling oneself in a virtual world. In this paper an experiment which was conducted in order to find the link between level of immersion and cyber sickness felt by participants is presented. Eighty-nine participants aged between 19 and 36 years have been equally divided into four groups with different level of VR immersion. The low-immersive group was represented by PC with monoscopic screen, the semi-immersive group was represented by CAVE with stereoscopic projector, the fully immersive group was represented by VR head-mounted display, and the last group was the control group without any kind of immersion. The task for the participants was to navigate through the maze for a specified amount of time (10 min). The Simulator Sickness Questionnaire was used as a subjective measure tool for cyber sickness level and Grooved Pegboard Test for assessing the fine dexterity, both before and after the experiment. Regarding the time spend in VR the fully immersive environment had the biggest problems as more than half of the participants had to stop before 10 min (p < 0.001). Concerning the cyber sickness, the significant increase in nausea score between pre-test and post-test scores has been observed in semi-immersive group (p = 0.0018) and fully immersive group (p < 0.0001). The increase in oculomotor score was smaller. The significant difference was noted only in fully immersive group (p = 0.0449). In spite of great nausea factor after the VR immersion the participants did not show a decrease of fine dexterity in any group (p < 0.001).
SiamAPN++: Siamese Attentional Aggregation Network for Real-Time UAV Tracking

Recently, the Siamese-based method has stood out from multitudinous tracking methods owing to its state-of-the-art (SOTA) performance. Nevertheless, due to various special challenges in UAV tracking, e.g., severe occlusion and fast motion, most existing Siamese-based trackers hardly combine superior performance with high efficiency. To this concern, in this paper, a novel attentional Siamese tracker (SiamAPN++) is proposed for real-time UAV tracking. By virtue of the attention mechanism, we conduct a special attentional aggregation network (AAN) consisting of self-AAN and cross-AAN for raising the representation ability of features eventually. The former AAN aggregates and models the self-semantic interdependencies of the single feature map via spatial and channel dimensions. The latter aims to aggregate the cross-interdependencies of two different semantic features including the location information of anchors. In addition, the anchor proposal network based on dual features is proposed to raise its robustness of tracking objects with various scales. Experiments on two well-known authoritative benchmarks are conducted, where SiamAPN++ outperforms its baseline SiamAPN and other SOTA trackers. Besides, real-world tests onboard a typical embedded platform demonstrate that SiamAPN++ achieves promising tracking results with real-time speed.
Advancement of the search process of salp swarm algorithm for global optimization problems

This paper propounds a modified version of the salp swarm algorithm (mSSA) for solving optimization problems more prolifically. This technique is refined from the base version with three simple but effective modifications. In the first one, the most important parameter in SSA responsible for balancing exploration and exploitation is chaotically changed by embedding a sinusoidal map in it to catch a better balance between exploration and exploitation from the first iteration until the last. As a short falling, SSA can’t exchange information amongst leaders of the chain. Therefore, a mutualistic relationship between two leader salps is included in mSSA to raise its search performance. Additionally, a random technique is systematically applied to the follower salps to introduce diversity in the chain. This can be since there may be some salps in the chain that do not necessarily follow the leader for exploring unvisited areas of the search space. Several test problems are solved by the advocated approach and results are presented in comparison with the relevant results in the available literature. It is ascertained that mSSA, despite its simplicity, significantly outperforms not only the basic SSA but also numerous recent algorithms in terms of fruitful solution precision and convergent trend line.
Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback

Conversational interfaces for the detail-oriented retail fashion domain are more natural, expressive, and user friendly than classical keyword-based search interfaces. In this paper, we introduce the Fashion IQ dataset to support and advance research on interactive fashion image retrieval. Fashion IQ is the first fashion dataset to provide human-generated captions that distinguish similar pairs of garment images together with side-information consisting of real-world product descriptions and derived visual attribute labels for these images. We provide a detailed analysis of the characteristics of the Fashion IQ data, and present a transformer-based user simulator and interactive image retriever that can seamlessly integrate visual attributes with image features, user feedback, and dialog history, leading to improved performance over the state of the art in dialog-based image retrieval. We believe that our dataset will encourage further work on developing more natural and real-world applicable conversational shopping assistants. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentation

Recently, referring image segmentation has aroused widespread interest. Previous methods perform the multi-modal fusion between language and vision at the decoding side of the network. And, linguistic feature interacts with visual feature of each scale separately, which ignores the continuous guidance of language to multi-scale visual features. In this work, we propose an encoder fusion network (EFN), which transforms the visual encoder into a multi-modal feature learning network, and uses language to refine the multi-modal features progressively. Moreover, a co-attention mechanism is embedded in the EFN to realize the parallel update of multi-modal features, which can promote the consistent of the cross-modal information representation in the semantic space. Finally, we propose a boundary enhancement module (BEM) to make the network pay more attention to the fine structure. The experiment results on four benchmark datasets demonstrate that the proposed approach achieves the state-of-the-art performance under different evaluation metrics without any post-processing.
HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features

We present HoHoNet, a versatile and efficient framework for holistic understanding of an indoor 360-degree panorama using a Latent Horizontal Feature (LHFeat). The compact LHFeat flattens the features along the vertical direction and has shown success in modeling per-column modality for room layout reconstruction. HoHoNet advances in two important aspects. First, the deep architecture is redesigned to run faster with improved accuracy. Second, we propose a novel horizon-to-dense module, which relaxes the per-column output shape constraint, allowing per-pixel dense prediction from LHFeat. HoHoNet is fast: It runs at 52 FPS and 110 FPS with ResNet-50 and ResNet-34 backbones respectively, for modeling dense modalities from a high-resolution 512 × 1024 panorama. HoHoNet is also accurate. On the tasks of layout estimation and semantic segmentation, HoHoNet achieves results on par with current state-of-the-art. On dense depth estimation, HoHoNet outperforms all the prior arts by a large margin. Code is available at https://github.com/sunset1995/HoHoNet.
Distilling Causal Effect of Data in Class-Incremental Learning

We propose a causal framework to explain the catastrophic forgetting in Class-Incremental Learning (CIL) and then derive a novel distillation method that is orthogonal to the existing anti-forgetting techniques, such as data replay and feature/label distillation. We first 1) place CIL into the framework, 2) answer why the forgetting happens: the causal effect of the old data is lost in new training, and then 3) explain how the existing techniques mitigate it: they bring the causal effect back. Based on the causal framework, we propose to distill the Colliding Effect between the old and the new data, which is fundamentally equivalent to the causal effect of data replay, but without any cost of replay storage. Thanks to the causal effect analysis, we can further capture the Incremental Momentum Effect of the data stream, removing which can help to retain the old effect overwhelmed by the new data effect, and thus alleviate the forgetting of the old class in testing. Extensive experiments on three CIL benchmarks: CIFAR-100, ImageNet-Sub&Full, show that the proposed causal effect distillation can improve various state-of-the-art CIL methods by a large margin (0.72%–9.06%). <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Image-to-image Translation via Hierarchical Style Disentanglement

Recently, image-to-image translation has made significant progress in achieving both multi-label (i.e., translation conditioned on different labels) and multi-style (i.e., generation with diverse styles) tasks. However, due to the unexplored independence and exclusiveness in the labels, existing endeavors are defeated by involving uncontrolled manipulations to the translation results. In this paper, we propose Hierarchical Style Disentanglement (HiSD) to address this issue. Specifically, we organize the labels into a hierarchical tree structure, in which independent tags, exclusive attributes, and disentangled styles are allocated from top to bottom. Correspondingly, a new translation process is designed to adapt the above structure, in which the styles are identified for controllable translations. Both qualitative and quantitative results on the CelebA-HQ dataset verify the ability of the proposed HiSD. The code has been released at https://github.com/imlixinyang/HiSD.
DoDNet: Learning to Segment Multi-Organ and Tumors from Multiple Partially Labeled Datasets

Due to the intensive cost of labor and expertise in annotating 3D medical images at a voxel level, most benchmark datasets are equipped with the annotations of only one type of organs and/or tumors, resulting in the so-called partially labeling issue. To address this issue, we propose a dynamic on-demand network (DoDNet) that learns to segment multiple organs and tumors on partially labeled datasets. DoD-Net consists of a shared encoder-decoder architecture, a task encoding module, a controller for dynamic filter generation, and a single but dynamic segmentation head. The information of current segmentation task is encoded as a task-aware prior to tell the model what the task is expected to achieve. Different from existing approaches which fix kernels after training, the kernels in dynamic head are generated adaptively by the controller, conditioned on both input image and assigned task. Thus, DoDNet is able to segment multiple organs and tumors, as done by multiple networks or a multi-head network, in a much efficient and flexible manner. We created a large-scale partially labeled dataset called MOTS and demonstrated the superior performance of our DoDNet over other competitors on seven organ and tumor segmentation tasks. We also transferred the weights pre-trained on MOTS to a downstream multi-organ segmentation task and achieved state-of-the-art performance. This study provides a general 3D medical image segmentation model that has been pre-trained on a large-scale partially labeled dataset and can be extended (after fine-tuning) to downstream volumetric medical data segmentation tasks. Code and models are available at: https://git.io/DoDNet
FISS GAN: A Generative Adversarial Network for Foggy Image Semantic Segmentation

Because pixel values of foggy images are irregularly higher than those of images captured in normal weather (clear images), it is difficult to extract and express their texture. No method has previously been developed to directly explore the relationship between foggy images and semantic segmentation images. We investigated this relationship and propose a generative adversarial network (GAN) for foggy image semantic segmentation (FISS GAN), which contains two parts: an edge GAN and a semantic segmentation GAN. The edge GAN is designed to generate edge information from foggy images to provide auxiliary information to the semantic segmentation GAN. The semantic segmentation GAN is designed to extract and express the texture of foggy images and generate semantic segmentation images. Experiments on foggy cityscapes datasets and foggy driving datasets indicated that FISS GAN achieved state-of-the-art performance.
A novel image encryption algorithm based on SPWLCM and DNA coding

Using chaotic system for image encryption is a feasible and popular research interest. In this paper, a compound Sine-Piecewise Linear Chaotic Map is proposed to improve the chaotic characteristics of the original chaotic map. Compared with the original chaotic system, the proposed compound chaotic system has better dynamics performances, larger parameter space, which is more suitable for cryptographic applications. Furthermore, an image encryption algorithm based on this chaotic model and varying DNA coding is proposed. In this encryption algorithm, a novel method is provided to convert decimal numbers to binary numbers. Several numerical experiments show that the proposed encryption algorithm has good security performances, and is competitive with other chaos-based image encryption algorithms.
Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: Theory, implementation and analysis on standard tasks

The recently proposed VBx diarization method uses a Bayesian hidden Markov model to find speaker clusters in a sequence of x-vectors. In this work we perform an extensive comparison of performance of the VBx diarization with other approaches in the literature and we show that VBx achieves superior performance on three of the most popular datasets for evaluating diarization: CALLHOME, AMI and DIHARD II datasets. Further, we present for the first time the derivation and update formulae for the VBx model, focusing on the efficiency and simplicity of this model as compared to the previous and more complex BHMM model working on frame-by-frame standard Cepstral features. Together with this publication, we release the recipe for training the x-vector extractors used in our experiments on both wide and narrowband data, and the VBx recipes that attain state-of-the-art performance on all three datasets. Besides, we point out the lack of a standardized evaluation protocol for AMI dataset and we propose a new protocol for both Beamformed and Mix-Headset audios based on the official AMI partitions and transcriptions.
A comprehensive survey of LIDAR-based 3D object detection methods with deep learning for autonomous driving

LiDAR-based 3D object detection for autonomous driving has recently drawn the attention of both academia and industry since it relies upon a sensor that incorporates appealing features like insensitivity to light and capacity to capture the 3D spatial structure of an object along with the continuous reduction of its purchase cost. Furthermore, the emergence of Deep Learning as the means to boost performance in 3D data analysis stimulated the production of a multitude of solutions for LIDAR-based 3D object detection which followed different approaches in an effort to respond effectively to several challenges. In view of this, this paper presents a comprehensive survey of LIDAR-based 3D object detection methods wherein an analysis of existing methods is addressed by taking into account a new categorisation that relies upon a common operational pipeline which describes the end-to-end functionality of each method. We next, discuss the existing benchmarking frameworks and present the performance achieved by each method in each of them. Finally, a discussion is presented that provides key insights aiming to capture the essence of current trends in LIDAR-based 3D object detection.
Performance analysis of cost-sensitive learning methods with application to imbalanced medical data

Many real-world machine learning applications require building models using highly imbalanced datasets. Usually, in medical datasets, the healthy patients or samples are dominant, making them the majority class, while the sick patients are few, making them the minority class. Researchers have proposed numerous machine learning methods to predict medical diagnosis. Still, the class imbalance problem makes it difficult for classifiers to adequately learn and distinguish between the minority and majority classes. Cost-sensitive learning and resampling techniques are used to deal with the class imbalance problem. This research focuses on developing robust cost-sensitive classifiers by modifying the objective functions of some well-known algorithms, such as logistic regression, decision tree, extreme gradient boosting, and random forest, which are then used to efficiently predict medical diagnosis. Meanwhile, as opposed to resampling techniques, our approach does not alter the original data distribution. Firstly, we implement the standard versions of these algorithms to provide a baseline for performance comparison. Secondly, we develop their corresponding cost-sensitive algorithms. For the proposed approaches, it is not necessary to change the distribution of the original data as the modified algorithms consider the imbalanced class distribution during training, thereby resulting in more reliable performance than when the data is resampled. Four popular medical datasets, including the Pima Indians Diabetes, Haberman Breast Cancer, Cervical Cancer Risk Factors, and Chronic Kidney Disease datasets, are used in the experiments to validate the performance of the proposed approach. The experimental results show that the cost-sensitive methods yield superior performance compared to the standard algorithms.
Deep learning: a statistical viewpoint

The remarkable practical success of deep learning has revealed some major surprises from a theoretical perspective. In particular, simple gradient methods easily find near-optimal solutions to non-convex optimization problems, and despite giving a near-perfect fit to training data without any explicit effort to control model complexity, these methods exhibit excellent predictive accuracy. We conjecture that specific principles underlie these phenomena: that overparametrization allows gradient methods to find interpolating solutions, that these methods implicitly impose regularization, and that overparametrization leads to benign overfitting, that is, accurate predictions despite overfitting training data. In this article, we survey recent progress in statistical learning theory that provides examples illustrating these principles in simpler settings. We first review classical uniform convergence results and why they fall short of explaining aspects of the behaviour of deep learning methods. We give examples of implicit regularization in simple settings, where gradient methods lead to minimal norm functions that perfectly fit the training data. Then we review prediction methods that exhibit benign overfitting, focusing on regression problems with quadratic loss. For these methods, we can decompose the prediction rule into a simple component that is useful for prediction and a spiky component that is useful for overfitting but, in a favourable setting, does not harm prediction accuracy. We focus specifically on the linear regime for neural networks, where the network can be approximated by a linear model. In this regime, we demonstrate the success of gradient flow, and we consider benign overfitting with two-layer networks, giving an exact asymptotic analysis that precisely demonstrates the impact of overparametrization. We conclude by highlighting the key challenges that arise in extending these insights to realistic deep learning settings.
Rain-Free and Residue Hand-in-Hand: A Progressive Coupled Network for Real-Time Image Deraining

Rainy weather is a challenge for many vision-oriented tasks ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , object detection and segmentation), which causes performance degradation. Image deraining is an effective solution to avoid performance drop of downstream vision tasks. However, most existing deraining methods either fail to produce satisfactory restoration results or cost too much computation. In this work, considering both effectiveness and efficiency of image deraining, we propose a progressive coupled network (PCNet) to well separate rain streaks while preserving rain-free details. To this end, we investigate the blending correlations between them and particularly devise a novel coupled representation module (CRM) to learn the joint features and the blending correlations. By cascading multiple CRMs, PCNet extracts the hierarchical features of multi-scale rain streaks, and separates the rain-free content and rain streaks progressively. To promote computation efficiency, we employ depth-wise separable convolutions and a U-shaped structure, and construct CRM in an asymmetric architecture to reduce model parameters and memory footprint. Extensive experiments are conducted to evaluate the efficacy of the proposed PCNet in two aspects: (1) image deraining on several synthetic and real-world rain datasets and (2) joint image deraining and downstream vision tasks ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , object detection and segmentation). Furthermore, we show that the proposed CRM can be easily adopted to similar image restoration tasks including image dehazing and low-light enhancement with competitive performance. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/kuijiang0802/PCNet</uri> .