Reviewerly/exHarmony huggingface dataset:
Comparing deep learning models for low-light natural scene image enhancement and their impact on object detection and classification: Overview, empirical evaluation, and challenges

Low-light image (LLI) enhancement is an important image processing task that aims at improving the illumination of images taken under low-light conditions. Recently, a remarkable progress has been made in utilizing deep learning (DL) approaches for LLI enhancement. This paper provides a concise and comprehensive review and comparative study of the most recent DL models used for LLI enhancement. To our knowledge, this is the first comparative study dedicated to DL-based models for LLI enhancement. We address LLI enhancement in two ways: (i) standalone, as a separate task, and (ii) end-to-end, as a pre-processing stage embedded within another high-level computer vision task, namely object detection and classification. The paper consists of six logical parts. First, we provide an overview of the background and literature in LLI enhancement. Second, we describe the test data and experimental setup of the study. Third, we present a quantitative and qualitative comparison of the visual and perceptual quality achieved by 10 of the most recent DL-based LLI enhancement models. Fourth, we present a comparative analysis for object detection and classification performance achieved by 4 different object detection models applied on LLIs and their enhanced counterparts. Fifth, we perform a feature analysis of DL feature maps extracted from normal, low-light, and enhanced images, and perform the occlusion experiment to better understand the effect of LLI enhancement on the object detection and classification task. Finally, we provide our conclusions and highlight future steps and potential directions. • Evaluates Deep Learning (DL) models for Low-light Image (LLI) enhancement. • Compares 10 LLI enhancement models and 4 object detection and classification models. • Provides a quantitative and qualitative comparison of visual and perceptual quality. • Evaluates impact of LLI enhancement on object detecting and classification quality. • Performs occlusion experiment to study LLI enhancement’s effect on object detection.
Human Activity Recognition: Review, Taxonomy and Open Challenges

Nowadays, Human Activity Recognition (HAR) is being widely used in a variety of domains, and vision and sensor-based data enable cutting-edge technologies to detect, recognize, and monitor human activities. Several reviews and surveys on HAR have already been published, but due to the constantly growing literature, the status of HAR literature needed to be updated. Hence, this review aims to provide insights on the current state of the literature on HAR published since 2018. The ninety-five articles reviewed in this study are classified to highlight application areas, data sources, techniques, and open research challenges in HAR. The majority of existing research appears to have concentrated on daily living activities, followed by user activities based on individual and group-based activities. However, there is little literature on detecting real-time activities such as suspicious activity, surveillance, and healthcare. A major portion of existing studies has used Closed-Circuit Television (CCTV) videos and Mobile Sensors data. Convolutional Neural Network (CNN), Long short-term memory (LSTM), and Support Vector Machine (SVM) are the most prominent techniques in the literature reviewed that are being utilized for the task of HAR. Lastly, the limitations and open challenges that needed to be addressed are discussed.
Multifeature Fusion-Based Object Detection for Intelligent Transportation Systems

The detection of 3D objects with high precision from point cloud data has become a crucial research topic in intelligent transportation systems. By effectively modeling global and local features, it can be acquired the state-of-the-art detector for 3D object detection. Nevertheless, regarding the previous work on feature representations, volumetric generation or point learning methods have difficulty building the relationships between local features and global features. Thus, we propose a multi-feature fusion network (MFFNet) to improve detection precision for 3D point cloud data by combining the global features from 3D voxel convolutions with the local features from the point learning network. Our algorithm is an end-to-end detection framework that contains a voxel convolutional module, a local point feature module and a detection head. Significantly, MFFNet constructs the local point feature set with point learning and sampling and the global feature map through 3D voxel convolution from raw point clouds. The detection head can use the obtained fusion feature to predict the position and category of the examined 3D object, so the proposed method can obtain higher precision than existing approaches. An experimental evaluation on the KITTI 3D object detection dataset obtain 97% MAP (Mean Average Precision) and Waymo Open dataset obtain 80% MAP, which proves the efficiency of the developed feature fusion representation method for 3D objects, and it can achieve satisfactory location accuracy.
Intelligent Hybrid Deep Learning Model for Breast Cancer Detection

Breast cancer (BC) is a type of tumor that develops in the breast cells and is one of the most common cancers in women. Women are also at risk from BC, the second most life-threatening disease after lung cancer. The early diagnosis and classification of BC are very important. Furthermore, manual detection is time-consuming, laborious work, and, possibility of pathologist errors, and incorrect classification. To address the above highlighted issues, this paper presents a hybrid deep learning (CNN-GRU) model for the automatic detection of BC-IDC (+,−) using whole slide images (WSIs) of the well-known PCam Kaggle dataset. In this research, the proposed model used different layers of architectures of CNNs and GRU to detect breast IDC (+,−) cancer. The validation tests for quantitative results were carried out using each performance measure (accuracy (Acc), precision (Prec), sensitivity (Sens), specificity (Spec), AUC and F1-Score. The proposed model shows the best performance measures (accuracy 86.21%, precision 85.50%, sensitivity 85.60%, specificity 84.71%, F1-score 88%, while AUC 0.89 which overcomes the pathologist’s error and miss classification problem. Additionally, the efficiency of the proposed hybrid model was tested and compared with CNN-BiLSTM, CNN-LSTM, and current machine learning and deep learning (ML/DL) models, which indicated that the proposed hybrid model is more robust than recent ML/DL approaches.
Blockchain Technology for Data Privacy using Contract Mechanism for 5G Networks

With ever-evolving internet models automating and digitizing innumerable industrial and domestic applications, today&#x2019;s world has achieved tremendous gains in the exchange of healthcare data and the initiation of essential measures. Private information and data integrity are the two most important considerations in today&#x2019;s healthcare data-sharing environment. Encryption is critical when it comes to health records and other sensitive data in order to prevent unauthorized parties from seeing or accessing the data. There may be no feasible method to deploy typical encryption algorithms through the intermediary node, and the intermediary nodes should not be given excessive weight. Blockchain technology may be used to encrypt data between nodes in the 5G network&#x2019;s design, and this article addresses the different security difficulties that exist in the current mechanism and the current strategies for IoT-driven healthcare monitoring frameworks&#x2019; security solutions. The proposed approach was evaluated using a variety of performance parameters, and the findings indicated that it outperformed the alternatives.
EFR-CSTP: Encryption for face recognition based on the chaos and semi-tensor product theory

Face recognition is a relatively common method humans use to confirm their identity. Protecting the safe transmission of face images has become a hot issue. This paper proposes an encryption method for face images, EFR-CSTP. The EFR-CSTP is very efficient because it only encrypts the facial in the face image. Firstly, the secret keys are generated by Hash. They are the parameters and the initial values of the two-dimensional logistic tent modular map (2D-LTMM), the chaotic sequence required for encryption by 2D-LTMM. Secondly, facial information is recognized by the histogram of oriented gradients (HOG). Note that the entire algorithm only encrypts this facial image. Finally, the facial image is used as the input of the EFR-CSTP, and the ciphertext image is generated by scrambling, STP combined with the XOR diffusion method. We evaluated our algorithm in the IMDB-WIKI dataset. We also evaluate the EFR-CSTP and other algorithms on the same datasets. The evaluation results show that the EFR-CSTP has excellent performance and efficiency.
Chaotic marine predators algorithm for global optimization of real-world engineering problems

A novel metaheuristic called Chaotic Marine Predators Algorithm (CMPA) is proposed and investigated for the optimization of engineering problems. CMPA integrates the exploration merits of the recently proposed Marine Predators Algorithm (MPA) with the chaotic maps exploitation capabilities. Several chaotic maps were applied in the proposed CMPA to govern MPA parameters that eventually led to controlled exploration and exploitation of search. This study makes an initial attempt to explore and employ CMPA in decoding complex and challenging design and manufacturing problems. For performance evaluation of the proposed algorithm, CEC 2020 numerical problems having different dimensions and five widely adopted constrained design problems were solved. For all problems, both qualitative and qualitative results are examined and discussed. Moreover, two case studies of multi-pass turning were examined by the proposed CMPA algorithm to optimize the cutting operation with a minimum cost of production per unit objective. Furthermore, the suggested CMPA algorithm has been investigated for solving a real-world structural topology optimization problem. Statistical analysis is performed, and the results of CMPA are compared with twelve distinguished algorithms. Outcomes of the proposed variant algorithm on the benchmarks demonstrate its significantly improved performance relative to other optimizers including a variant of MPA and two state-of-the-art IEEE CEC competitions winners algorithms. Findings from the manufacturing process exhibit CMPA proficiency in solving arduous real-world design problems.
Cross-Image Relational Knowledge Distillation for Semantic Segmentation

Current Knowledge Distillation (KD) methods for semantic segmentation often guide the student to mimic the teacher's structured information generated from individual data samples. However, they ignore the global semantic relations among pixels across various images that are valuable for KD. This paper proposes a novel Cross-Image Relational KD (CIRKD), which focuses on transferring structured pixel-to-pixel and pixel-to-region relations among the whole images. The motivation is that a good teacher network could construct a well-structured feature space in terms of global pixel dependencies. CIRKD makes the student mimic better structured semantic relations from the teacher, thus improving the segmentation performance. Experimental results over Cityscapes, CamVid and Pascal VOC datasets demonstrate the effectiveness of our proposed approach against state-of-the-art distillation methods. The code is available at https://github.com/winycg/CIRKD.
Equalized Focal Loss for Dense Long-Tailed Object Detection

Despite the recent success of long-tailed object detection, almost all long-tailed object detectors are developed based on the two-stage paradigm. In practice, one-stage detectors are more prevalent in the industry because they have a simple and fast pipeline that is easy to deploy. However, in the long-tailed scenario, this line of work has not been explored so far. In this paper, we investigate whether one-stage detectors can perform well in this case. We discover the primary obstacle that prevents one-stage detectors from achieving excellent performance is: categories suffer from different degrees of positive-negative imbalance problems under the long-tailed data distribution. The conventional focal loss balances the training process with the same modulating factor for all categories, thus failing to handle the long-tailed problem. To address this issue, we propose the Equalized Focal Loss (EFL) that rebalances the loss contribution of positive and negative samples of different categories independently according to their imbalance degrees. Specifically, EFL adopts a category-relevant modulating factor which can be adjusted dynamically by the training status of different categories. Extensive experiments conducted on the challenging LVIS v1 benchmark demonstrate the effectiveness of our proposed method. With an end-to-end training pipeline, EFL achieves 29.2% in terms of overall AP and obtains significant performance improvements on rare categories, surpassing all existing state-of-the-art methods. The code is available at https: //github.com/ModelTC/EOD.
Reusing the Task-specific Classifier as a Discriminator: Discriminator-free Adversarial Domain Adaptation

Adversarial learning has achieved remarkable performances for unsupervised domain adaptation (UDA). Existing adversarial UDA methods typically adopt an additional discriminator to play the min-max game with a feature extractor. However, most of these methods failed to effectively leverage the predicted discriminative information, and thus cause mode collapse for generator. In this work, we address this problem from a different perspective and design a simple yet effective adversarial paradigm in the form of a discriminator-free adversarial learning network (DALN), wherein the category classifier is reused as a discriminator, which achieves explicit domain alignment and category distinguishment through a unified objective, enabling the DALN to leverage the predicted discriminative information for sufficient feature alignment. Basically, we introduce a Nuclear-norm Wasserstein discrepancy (NWD) that has definite guidance meaning for performing discrimination. Such NWD can be coupled with the classifier to serve as a discriminator satisfying the K-Lipschitz constraint without the requirements of additional weight clipping or gradient penalty strategy. Without bells and whistles, DALN compares favorably against the existing state-of-the-art (SOTA) methods on a variety of public datasets. Moreover, as a plug-and-play technique, NWD can be directly used as a generic regularizer to benefit existing UDA algorithms. Code is available at https://github.com/xiaoachen98/DALN.
Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identification

Recent studies show that, both explicit deep feature matching as well as large-scale and diverse training data can significantly improve the generalization of person reidentification. However, the efficiency of learning deep matchers on large-scale data has not yet been adequately studied. Though learning with classification parameters or class memory is a popular way, it incurs large memory and computational costs. In contrast, pairwise deep metric learning within mini batches would be a better choice. However, the most popular random sampling method, the well-known PKsampler, is not informative and efficient for deep metric learning. Though online hard example mining has improved the learning efficiency to some extent, the mining in mini batches after random sampling is still limited. This inspires us to explore the use of hard example mining earlier, in the data sampling stage. To do so, in this paper, we propose an efficient mini-batch sampling method, called graph sampling (GS), for large-scale deep metric learning. The basic idea is to build a nearest neighbor relationship graph for all classes at the beginning of each epoch. Then, each mini batch is composed of a randomly selected class and its nearest neighboring classes so as to provide informative and challenging examples for learning. Together with an adapted competitive baseline, we improve the state of the art in generalizable person re-identification significantly, by 25.1% in Rank-1 on MSMT17 when trained on RandPerson. Besides, the proposed method also outperforms the competitive baseline, by 6.8% in Rank-1 on CUHK03-NP when trained on MSMT17. Meanwhile, the training time is significantly reduced, from 25.4 hours to 2 hours when trained on RandPerson with 8,000 identities. Code is available at https://github.com/ShengcaiLiao/QAConv.
Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation

Weakly Supervised Semantic Segmentation (WSSS) based on image-level labels has attracted much attention due to low annotation costs. Existing methods often rely on Class Activation Mapping (CAM) that measures the correlation between image pixels and classifier weight. However, the classifier focuses only on the discriminative regions while ignoring other useful information in each image, resulting in incomplete localization maps. To address this issue, we propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Specifically, IPE tailors prototypes for every image to capture complete regions, formed our Image-Specific CAM (IS-CAM), which is realized by two sequential steps. In addition, GSC is proposed to construct the consistency of general CAM and our specific IS-CAM, which further optimizes the feature representation and empowers a self-correction ability of prototype exploration. Extensive experiments are conducted on PASCAL VOC 2012 and MS COCO 2014 segmentation benchmark and results show our SIPE achieves new state-of-the-art performance using only image-level labels. The code is available at https://github.com/chenqi1126/SIPE.
Revisiting Random Channel Pruning for Neural Network Compression

Channel (or 3D filter) pruning serves as an effective way to accelerate the inference of neural networks. There has been a flurry of algorithms that try to solve this practical problem, each being claimed effective in some ways. Yet, a benchmark to compare those algorithms directly is lacking, mainly due to the complexity of the algorithms and some custom settings such as the particular network configuration or training procedure. A fair benchmark is important for the further development of channel pruning. Meanwhile, recent investigations reveal that the channel configurations discovered by pruning algorithms are at least as important as the pre-trained weights. This gives channel pruning a new role, namely searching the optimal channel configuration. In this paper, we try to determine the channel configuration of the pruned models by random search. The proposed approach provides a new way to compare different methods, namely how well they behave compared with random pruning. We show that this simple strategy works quite well compared with other channel pruning methods. We also show that under this setting, there are surprisingly no clear winners among different channel importance evaluation methods, which then may tilt the research efforts into advanced channel configuration searching methods. Code will be released at https://github.com/ofsoundof/random_channel_pruning.
An Image Patch is a Wave: Phase-Aware Vision MLP

In the field of computer vision, recent works show that a pure MLP architecture mainly stacked by fully-connected layers can achieve competing performance with CNN and transformer. An input image of vision MLP is usually split into multiple tokens (patches), while the existing MLP models directly aggregate them with fixed weights, neglecting the varying semantic information of tokens from different images. To dynamically aggregate tokens, we propose to represent each token as a wave function with two parts, amplitude and phase. Amplitude is the original feature and the phase term is a complex value changing according to the semantic contents of input images. Introducing the phase term can dynamically modulate the relationship between tokens and fixed weights in MLP. Based on the wave-like token representation, we establish a novel Wave-MLP architecture for vision tasks. Extensive experiments demonstrate that the proposed Wave-MLP is superior to the state-of-the-art MLP architectures on various vision tasks such as image classification, object detection and semantic segmentation. The source code is available at https://github.com/huawei-noah/CV-Backbones/tree/master/wavemlp_pytorch and https://gitee.com/mindspore/models/tree/master/research/cv/wave_mlp.
Learning Multiple Adverse Weather Removal via Two-stage Knowledge Learning and Multi-contrastive Regularization: Toward a Unified Model

In this paper, an ill-posed problem of multiple adverse weather removal is investigated. Our goal is to train a model with a ‘unified’ architecture and only one set of pretrained weights that can tackle multiple types of adverse weathers such as haze, snow, and rain simultaneously. To this end, a two-stage knowledge learning mechanism including knowledge collation (KC) and knowledge examination (KE) based on a multi-teacher and student architecture is proposed. At the KC, the student network aims to learn the comprehensive bad weather removal problem from multiple well-trained teacher networks where each of them is specialized in a specific bad weather removal problem. To accomplish this process, a novel collaborative knowledge transfer is proposed. At the KE, the student model is trained without the teacher networks and examined by challenging pixel loss derived by the ground truth. Moreover, to improve the performance of our training framework, a novel loss function called multi-contrastive knowledge regularization (MCR) loss is proposed. Experiments on several datasets show that our student model can achieve promising results on different bad weather removal tasks simultaneously. The code is available in our project page.
Autonomous Behavioral Decision for Vehicular Agents Based on Cyber-Physical Social Intelligence

In future smart cities supported by cyber-physical social intelligence, autonomous behavioral decision for vehicular agents is going to become a general demand. Despite much progress achieved in autonomous behavioral decision of vehicular agents, the existing works can just be used in scenarios of short-distance behavioral decision. Naturally, they are not well suitable for long-distance behavioral decision tasks, posing much challenge in realistic cyber-physical environment. To bridge the existing gaps, this article proposes an autonomous behavioral decision framework for vehicular agents using cyber-physical social intelligence. First, it is expected to establish a dynamic planning model with multiple objectives and constraints. This can be embedded into the control unit of a vehicular agent to endow it with proper social intelligence. On this basis, an iterative search algorithm is specifically designed for it to find the optimal solutions from the whole solution space. Finally, two typical situation cases are implemented with use of simulation modeling to display the working architecture of the proposed method. In addition, a universal optimization search algorithm is selected as the baseline to be compared with the proposed method. The comparison results reveal both planning utility and running efficiency of the proposed method.