Reviewerly/exHarmony huggingface dataset:
SHT-based public auditing protocol with error tolerance in FDL-empowered IoVs

With the intelligentization of the Internet of Vehicles (IoVs), Artificial Intelligence (AI) technology is becoming more and more essential, especially deep learning. Federated Deep Learning (FDL) is a novel distributed machine learning technology and is able to address the challenges like data security, privacy risks, and huge communication overheads from big raw data sets. However, FDL can only guarantee data security and privacy among multiple clients during data training. If the data sets stored locally in clients are corrupted, including being tampered with and lost, the training results of the FDL in intelligent IoVs must be negatively affected. In this paper, we are the first to design a secure data auditing protocol to guarantee the integrity and availability of data sets in FDL-empowered IoVs. Specifically, the cuckoo filter and Reed-Solomon codes are utilized to guarantee error tolerance, including efficient corrupted data locating and recovery. In addition, a novel data structure, Skip Hash Table (SHT) is designed to optimize data dynamics. Finally, we illustrate the security of the scheme with the Computational Diffie-Hellman (CDH) assumption on bilinear groups. Sufficient theoretical analyses and performance evaluations demonstrate the security and efficiency of our scheme for data sets in FDL-empowered IoVs.
A Recognizable Expression Line Portrait Synthesis Method in Portrait Rendering Robot

An artistic line portrait robot can generate, process, and draw line portraits. Compared to real face images, line portraits lose some recognizable information. Maintaining recognizability during the process of expression edition of line portraits is an important challenge for artistic portrait robots. A recognizable expression line portrait synthesis method based on a triangle coordinate system (TCS) is proposed. First, based on public facial expression databases JAFFE, Oulu CASIA, RaFD, and Cohn-Kanade (CK), by studying the feature deviations between different expressions of the same person, an expression deformation constraint criterion (EDCC) that is conducive to maintaining recognizable features is proposed. Then, by comparing features between the source line portrait and reference expression portrait, the expression features are calculated. Finally, under the EDCC, based on expression features, a recognizable expression line portrait is generated through image topological deformation based on TCS. In addition, we can synthesize different degrees of expression line portraits. On the public face datasets (FHHQ, CelebA-HQ, and CK), we implemented qualitative and quantitative contrast experiments. Experimental results demonstrate that this method can automatically synthesize an expression line portrait with reference expression, where the expression degree of the reference expression is controllable, and the generated expression portrait still has high recognizability. The expression samples generated by the proposed method are used for face authentication on the CK dataset, and only 0.22% of the samples fail to pass the authentication.
Model-Agnostic Meta-Learning for Multilingual Hate Speech Detection

Hate speech in social media is a growing phenomenon, and detecting such toxic content has recently gained significant traction in the research community. Existing studies have explored fine-tuning language models (LMs) to perform hate speech detection, and these solutions have yielded significant performance. However, most of these studies are limited to detecting hate speech only in English, neglecting the bulk of hateful content that is generated in other languages, particularly in low-resource languages. Developing a classifier that captures hate speech and nuances in a low-resource language with limited data is extremely challenging. To fill the research gap, we propose HateMAML, a model-agnostic meta-learning (MAML)-based framework that effectively performs hate speech detection in low-resource languages. HateMAML utilizes a self-supervision strategy to overcome the limitation of data scarcity and produces better LM initialization for fast adaptation to an unseen target language (i.e., cross-lingual transfer) or other hate speech datasets (i.e., domain generalization). Extensive experiments are conducted on five datasets across eight different low-resource languages. The results show that HateMAML outperforms the state-of-the-art baselines by more than 3% in the cross-domain multilingual transfer setting. We also conduct ablation studies to analyze the characteristics of HateMAML.
Underwater Color Disparities: Cues for Enhancing Underwater Images Toward Natural Color Consistencies

We observe that a natural image tends to exhibit similar histograms for color channels in the RGB color space and consistent statistical estimates for color channels in the Lab color space. We refer to these observations as natural color consistencies. In contrast, we discover that an underwater image does not always follow the natural color consistencies. Different color channels in an underwater image tend to give rise to very different distributions, regardless of whether the channels are in the RGB color space or Lab color space. We refer to these observations as underwater color disparities. To enhance an underwater image to make it appear more natural, it is necessary to correct its underwater color disparities to align with the natural color consistencies. To this end, we develop an adaptive attenuated channel compensation method based on optimal channel precorrection and a salient absorption map-guided fusion method for eliminating the color deviation in the RGB color space. We then develop a method to enhance the contrast of channel L and an adaptive color distribution specification method for improving the contrast and matching the color distribution in the Lab color space. Additionally, we develop an edge-enhanced mask fusion method for correcting blurry details. Our method is not a deep learning method but can effectively be applied to a single underwater image. The qualitative and quantitative empirical results validate that our method outperforms state-of-the-art underwater image enhancement methods. We release the reproducible code at https://gitee.com/wanghaoupc/Underwater Color Disparities for public evaluation.
Secure Federated Learning With Fully Homomorphic Encryption for IoT Communications

The emergence of the Internet of Things (IoT) has revolutionized people’s daily lives, providing superior quality services in cognitive cities, healthcare, and smart buildings. However, smart buildings use heterogeneous networks. The massive number of interconnected IoT devices increases the possibility of IoT attacks, emphasizing the necessity of secure and privacy-preserving solutions. Federated Learning (FL) has recently emerged as a promising Machine Learning (ML) paradigm for IoT networks to address these concerns. In FL, multiple devices collaborate to learn a global model without sharing their raw data. However, FL still faces privacy and security concerns due to the transmission of sensitive data (i.e., model parameters) over insecure communication channels. These concerns can be addressed using Fully Homomorphic Encryption (FHE), a powerful cryptographic technique that enables computations on encrypted data without requiring them to be decrypted first. In this study, we propose a secure FL approach in IoT-enabled smart cities that combines FHE and FL to provide secure data and maintain privacy in distributed environments. We present four different FL-based FHE approaches in which data are encrypted and transmitted over a secure medium. The proposed approaches achieved high accuracy, recall, precision, and F-scores, in addition to providing strong privacy and security safeguards. Furthermore, the proposed approaches effectively reduced communication overhead and latency compared to the baseline approach. These approaches yielded improvements ranging from 80.15% to 89.98% in minimizing communication overhead. Additionally, one of the approaches achieved a remarkable latency reduction of 70.38%. The implementation of these security models is non-trivial, and the code is publicly available at https://github.com/Artifitialleap-MBZUAI/Secure-Federated-Learning-with-Fully-Homomorphic-Encryption-for-IoT-Communications.
Newton-Raphson-based optimizer: A new population-based metaheuristic algorithm for continuous optimization problems

The Newton-Raphson-Based Optimizer (NRBO), a new metaheuristic algorithm, is suggested and developed in this paper. The NRBO is inspired by Newton-Raphson's approach, and it explores the entire search process using two rules: the Newton-Raphson Search Rule (NRSR) and the Trap Avoidance Operator (TAO) and a few groups of matrices to explore the best results further. The NRSR uses a Newton-Raphson method to improve the exploration ability of NRBO and increase the convergence rate to reach improved search space positions. The TAO helps the NRBO to avoid the local optima trap. The performance of NRBO was assessed using 64 benchmark numerical functions, including 23 standard benchmarks, 29 CEC2017 constrained benchmarks, and 12 CEC2022 benchmarks. In addition, the NRBO was employed to optimize 12 CEC2020 real-world constrained engineering optimization problems. The proposed NRBO was compared to seven state-of-the-art optimization algorithms, and the findings showed that the NRBO produced promising results due to its features, such as high exploration and exploitation balance, high convergence rate, and effective avoidance of local optima capabilities. In addition, the NRBO also validated on challenging wireless communication problem called the internet of vehicle problem, and the NRBO was able to find the optimal path for data transmission. Also, the performance of NRBO in training the deep reinforcement learning agents is also studied by considering the mountain car problem. The obtained results also proved the NRBO's excellent performance in handling challenging real-world engineering problems.
Efficient Quantum Image Classification Using Single Qubit Encoding

The domain of image classification has been seen to be dominated by high-performing deep-learning (DL) architectures. However, the success of this field, as seen over the past decade, has resulted in the complexity of modern methodologies scaling exponentially, commonly requiring millions of parameters. Quantum computing (QC) is an active area of research aimed toward greatly reducing problems of complexity faced in classical computing. With growing interest toward quantum machine learning (QML) for applications of image classification, many proposed algorithms require usage of numerous qubits. In the noisy intermediate-scale quantum (NISQ) era, these circuits may not always be feasible to execute effectively; therefore, we should aim to use each qubit as effectively and efficiently as possible, before adding additional qubits. This article proposes a new single-qubit-based deep quantum neural network for image classification that mimics traditional convolutional neural network (CNN) techniques, resulting in a reduced number of parameters compared with previous works. Our aim is to prove the concept of the initial proposal by demonstrating classification performance of the single-qubit-based architecture, as well as to provide a tested foundation for further development. To demonstrate this, our experiments were conducted using various datasets including MNIST, Fashion-MNIST, and ORL face datasets. To further our proposal in the context of the NISQ era, our experiments were intentionally conducted in noisy simulation environments. Initial test results appear promising, with classification accuracies of 94.6%, 89.5%, and 82.5% achieved on the subsets of MNIST, FMNIST, and ORL face datasets, respectively. In addition, proposals for further investigation and development were considered, where it is hoped that these initial results can be improved.
Potential sources of sensor data anomalies for autonomous vehicles: An overview from road vehicle safety perspective

Outstanding steps towards intelligent transportation systems with autonomous vehicles have been taken in the past few years. Nevertheless, the safety issue in autonomous vehicles is critical and remains to be fully solved. Sensor data provide information about the internal status of the system and the impact of its external environment, where the occurrence of sensor data anomalies indicates the existence of potential safety risks. Therefore, in this work, a taxonomy for potential sensor data anomaly sources from the perspective of road vehicle safety is proposed, motivated by the lack of a unified comprehensive taxonomy of sensor data anomaly identification for autonomous vehicles. In this context, sources are divided into; 1) fault or failure of the components or subsystems; 2) failure of the adaptability to the external environment; 3) cyber-attacks; and 4) faults or design deficiencies of sensors. Based on the taxonomy proposed, related works, and in particular, countermeasures for the four potential sources of sensor data anomalies in autonomous vehicles are then reviewed. In the context of providing a comprehensive discussion, other taxonomies of potential sources causing sensor data anomalies for autonomous vehicles and the issue of interpretability of sensor data anomalies are also discussed, providing insight into the strengths of the proposed taxonomy.
RoFormer: Enhanced transformer with Rotary Position Embedding

Position encoding has recently been shown to be effective in transformer architecture. It enables valuable supervision for dependency modeling between elements at different positions of the sequence. In this paper, we first investigate various methods to integrate positional information into the learning process of transformer-based language models. Then, we propose a novel method named Rotary Position Embedding (RoPE) to effectively leverage the positional information. Specifically, the proposed RoPE encodes the absolute position with a rotation matrix and meanwhile incorporates the explicit relative position dependency in the self-attention formulation. Notably, RoPE enables valuable properties, including the flexibility of sequence length, decaying inter-token dependency with increasing relative distances, and the capability of equipping linear self-attention with relative position encoding. Finally, we evaluate the enhanced transformer with rotary position embedding, also called RoFormer, on various long text classification benchmark datasets. Our experiments show that it consistently overcomes its alternatives. Furthermore, we provide a theoretical analysis to explain some experimental results. RoFormer is already integrated into Huggingface: https://huggingface.co/docs/transformers/model_doc/roformer.
Light Field Depth Estimation for Non-Lambertian Objects via Adaptive Cross Operator

Light field (LF) depth estimation is a crucial basis for LF-related applications. Most existing methods are based on the Lambertian assumption and cannot deal with non-Lambertian surfaces represented by transparent objects and mirrors. In this paper, we propose a novel Adaptive-Cross-Operator-based(ACO) depth estimation algorithm for non-Lambertian LF. By analyzing the imaging characteristics of non-Lambertian regions, it is found that the difficulty of depth estimation lies in the photo inconsistency of the center view. Combining with the two-branch structure, we propose ACO with an inter-branch cooperation strategy to adaptively separate depth information with different reflectance coefficients. We discover that the bimodal distribution feature of the operator filtering results can assist in the separation of multi-layer scene information. The first detection branch filters the EPI and implicitly records the severity of multi-layer scene aliasing. According to the identification of bimodal distribution features, the non-Lambertian regions are marked out and the depth of the foreground is estimated. The second branch receives guidance from the first to dynamically adjust the inner weight and infer the background’s depth after weakening the interference from the foreground. Finally, the depth information separation of multi-layer scenes is achieved by extracting the unique X-shaped linear structure. Without the reflection coefficients of the non-Lambertian object, the proposed method can produce high-quality depth estimation under the transparency of 90% to 20%. Experimental results show that the proposed ACO outperforms state-of-the-art LF depth estimation methods in terms of accuracy and robustness.
Tailoring Gradient Methods for Differentially Private Distributed Optimization

Decentralized optimization is gaining increased traction due to its widespread applications in large-scale machine learning and multi-agent systems. The same mechanism that enables its success, i.e., information sharing among participating agents, however, also leads to the disclosure of individual agents' private information, which is unacceptable when sensitive data are involved. As differential privacy is becoming a de facto standard for privacy preservation, recently results have emerged integrating differential privacy with distributed optimization. However, directly incorporating differential privacy design in existing distributed optimization approaches significantly compromises optimization accuracy. In this paper, we propose to redesign and tailor gradient methods for differentially-private distributed optimization, and propose two differential-privacy oriented gradient methods that can ensure both rigorous <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula> -differential privacy and optimality. The first algorithm is based on static-consensus based gradient methods, and the second algorithm is based on dynamic-consensus (gradient-tracking) based distributed optimization methods and, hence, is applicable to general directed interaction graph topologies. Both algorithms can simultaneously ensure almost sure convergence to an optimal solution and a finite privacy budget, even when the number of iterations goes to infinity. To our knowledge, this is the first time that both goals are achieved simultaneously. Numerical simulations using a distributed estimation problem and experimental results on a benchmark dataset confirm the effectiveness of the proposed approaches.
Dung beetle optimization algorithm based on quantum computing and multi-strategy fusion for solving engineering problems

The Dung beetle optimization algorithm is a kind of group intelligence optimization algorithm proposed by Jiankai Xue in 2022, which has the characteristics of strong optimization-seeking ability and fast convergence but suffers from the defect of easily falling into local optimum at the late stage of optimization-seeking as other group intelligence optimization algorithms. To address this problem, this paper proposes a dung beetle search algorithm (QHDBO) based on quantum computing and a multi-strategy hybrid. The good point set strategy is used to initialize the initial population of dung beetles . That makes the initial population more evenly distributed, and reduces the likelihood of the algorithm falling into a local optimum solution. The convergence factor and dynamic balance between the number of Spawning and foraging dung beetles is proposed. That allows the algorithm to focus on the global search in the early stages and local exploration in the later stages. The quantum computing based t-distribution variation strategy is used to variate the optimal global solution, that prevents the algorithm from falling into a local optimum. To verify the performance of the QHDBO algorithm, this paper compares QHDBO with six other swarm intelligence algorithms through 37 test functions and practical engineering application problems. The experimental results show that the improved dung beetle optimization algorithm significantly improves convergence speed and optimization accuracy and has good robustness.
SDIGRU: Spatial and Deep Features Integration Using Multilayer Gated Recurrent Unit for Human Activity Recognition

Smart video surveillance plays a significant role in public security via storing a huge amount of continual stream data, evaluates them, and generates warns where undesirable human activities are performed. Recognition of human activities in video surveillance has faced many challenges such as optimal evaluation of human activities under growing volumes of streaming data with complex computation and huge time processing complexity. To tackle these challenges we introduce a lightweighted spatial-deep features integration using multilayer GRU (SDIGRU). First, we extract spatial and deep features from frames sequence of realistic human activity videos via utilizing a lightweight MobileNetV2 model and then integrate those spatial-deep features. Although deep features can be used for human activity recognition, they contain only the high-level appearance, which is insufficient to correctly identify the particular activity of human. Thus, we jointly apply deep information with spatial appearance to produce detailed level information. Furthermore, we select rich informative features from spatial-deep appearances. Then, we train multilayer gated recurrent unit (GRU) and feed informative features to learn the temporal dynamics of human activity frames sequence at each time step of GRU. We conduct our experiments on benchmark YouTube11, HMDB51, and UCF101 datasets of human activity recognition. The empirical results show that our method achieved significant recognition performance with low computational complexity and quick response. Finally, we compare the results with existing state-of-the-art techniques, which show the effectiveness of our method.
CVANet: Cascaded visual attention network for single image super-resolution

Deep convolutional neural networks (DCNNs) have exhibited excellent feature extraction and detail reconstruction capabilities for single image super-resolution (SISR). Nevertheless, most previous DCNN-based methods do not fully utilize the complementary strengths between feature maps, channels, and pixels. Therefore, it hinders the ability of DCNNs to represent abundant features. To tackle the aforementioned issues, we present a Cascaded Visual Attention Network for SISR called CVANet, which simulates the visual attention mechanism of the human eyes to focus on the reconstruction process of details. Specifically, we first designed a trainable feature attention module (FAM) for feature-level attention learning. Afterward, we introduce a channel attention module (CAM) to reinforce feature maps under channel-level attention learning. Meanwhile, we propose a pixel attention module (PAM) that adaptively selects representative features from the previous layers, which are utilized to generate a high-resolution image. Satisfactory, our CVANet can effectively improve the resolution of images by exploring the feature representation capabilities of different modules and the visual perception properties of the human eyes. Extensive experiments with different methods on four benchmarks demonstrate that our CVANet outperforms the state-of-the-art (SOTA) methods in subjective visual perception, PSNR, and SSIM.The code will be made available https://github.com/WilyZhao8/CVANet.