Reviewerly/exHarmony huggingface dataset:
Multi-operator opposition-based learning with the neighborhood structure for numerical optimization problems and its applications

Opposition-based learning (OBL) is an effective strategy that adjusts the population to accelerate the convergence of the algorithm. However, OBL involves two phases (generation and calculation) that are seldom mentioned simultaneously. Besides, individual information in OBL is not fully used to guide evolution. First, a novel neighborhood generation strategy is proposed to fully use individual information. There are numerous variants of the neighborhood structure. Three prominent neighborhood structures are designed to demonstrate their benefits in various dimensions. Second, a multi-operator calculation method with two distributions is introduced to reach the equilibrium between diversification and intensification. Consequently, this paper proposes three multi-operator neighborhood-based OBL variants (MNOBLs). Compared with the traditional OBL variants, MNOBLs generate the opposite population by using a neighborhood of the individuals. A well-balanced calculation method can lead to good convergence behavior. Subsequently, the proposed MNOBLs are embedded into the basic differential evolution (DE) algorithm to obtain the proposed algorithms. The extensive and rigorous numerical experiments are designed at three levels: strategies, algorithms, and functions. Specifically, the effectiveness and efficiency of the proposed algorithms are verified by comparing them with 13 state-of-the-art DE variants and four metaheuristics on the CEC 2017, and CEC 2022 test suites, and two real-world optimization problems. The comprehensive results show that the proposed OBL variants provide promising results for most problems.
Explainability for Large Language Models: A Survey

Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transparency poses unwanted risks for downstream applications. Therefore, understanding and explaining these models is crucial for elucidating their behaviors, limitations, and social impacts. In this article, we introduce a taxonomy of explainability techniques and provide a structured overview of methods for explaining Transformer-based language models. We categorize techniques based on the training paradigms of LLMs: traditional fine-tuning-based paradigm and prompting-based paradigm. For each paradigm, we summarize the goals and dominant approaches for generating local explanations of individual predictions and global explanations of overall model knowledge. We also discuss metrics for evaluating generated explanations and discuss how explanations can be leveraged to debug models and improve performance. Lastly, we examine key challenges and emerging opportunities for explanation techniques in the era of LLMs in comparison to conventional deep learning models.
Balancing Innovation and Security in the Cloud

This chapter delves into the intricate relationship between innovation and security in the digital age. It highlights the challenges of the digital age and the transformative impact of cloud computing, emphasizing its role in driving innovation. The chapter also delves into cloud service and deployment models, discussing the benefits and challenges of cloud security. It also discusses the role of innovation in driving progress through case studies and addressing challenges organizations face. The chapter also discusses risk assessment and mitigation in the cloud, compliance, regulatory challenges, legal and ethical considerations, and best practices. It also explores emerging technologies like AI and machine learning, Zero Trust security, and future directions.
Autoencoders and their applications in machine learning: a survey

Abstract Autoencoders have become a hot researched topic in unsupervised learning due to their ability to learn data features and act as a dimensionality reduction method. With rapid evolution of autoencoder methods, there has yet to be a complete study that provides a full autoencoders roadmap for both stimulating technical improvements and orienting research newbies to autoencoders. In this paper, we present a comprehensive survey of autoencoders, starting with an explanation of the principle of conventional autoencoder and their primary development process. We then provide a taxonomy of autoencoders based on their structures and principles and thoroughly analyze and discuss the related models. Furthermore, we review the applications of autoencoders in various fields, including machine vision, natural language processing, complex network, recommender system, speech process, anomaly detection, and others. Lastly, we summarize the limitations of current autoencoder algorithms and discuss the future directions of the field.
Variational quantum time evolution without the quantum geometric tensor

Real- and imaginary-time quantum state evolutions are crucial in physics and chemistry for exploring quantum dynamics, preparing ground states, and computing thermodynamic observables. On near-term devices, variational quantum time evolution is a promising candidate for these tasks, as the required circuit model can be tailored to the available devices' capabilities. Due to the evaluation of the quantum geometric tensor (QGT), however, this approach quickly becomes infeasible for relevant system sizes. Here, we propose a dual formulation for variational time evolution, which replaces the calculation of the QGT by solving a fidelity-based optimization to compute updates to the dynamics in each time step. We demonstrate our algorithm for the time evolution of the Heisenberg Hamiltonian and show that it accurately reproduces the system dynamics at a fraction of the cost of standard variational quantum time evolution algorithms.
Analysis on unit maximum capacity of orthogonal multiple watermarking for multimedia signals in B5G wireless communications

B5G aims to meet the demands of today's growing mobile traffic and expand the communication space. Considering that the intelligent application to B5G wireless communications will involve the security issues of user data and operational data, in this paper, the maximum capacity problem of the multi-watermarking method in multimedia signal hiding is studied to alleviate the information security problem of B5G. In the analyzed multi-watermarking process, the Spread Transform Dither Modulation is employed. During the watermarking procedures, through Gram-Schmidt orthogonalization, we can obtain the multiple spreading vectors. Consequently, the multiple watermarks can be simultaneously embedded into the same position of multimedia signal. Moreover, the multiple watermarks can be extracted without affecting one another during the extraction process. In our research, we intensively analyze the effect of the size of the spreading vector on the unit maximum capacity; consequently, we discover the hidden theoretical relationship between the size of the spreading vector and the unit maximum capacity. Many experiments have been conducted to seek out the optimal parameter values, with which the method can achieve the highest robustness, on the premise of possessing high capacity and good imperceptibility.
Transformer Driven Matching Selection Mechanism for Multi-Label Image Classification

Graph Matching has recently emerged as an attractive technique applied to various computer vision tasks. Graph Matching based multi-label image classification, in particular, treats each image as a bag of instances and reformulates the classification task as an instance-label matching selection problem, achieving state-of-the-art results on diverse benchmarks. However, the generalization and scalability of such learned model cannot be well guaranteed due to its manually predetermined graph structure and high-dimension embedding of dense connections between instances and labels. To address these limitations, in this work, we propose a novel Transformer Driven Matching Selection framework for Multi-Label Image Classification (C-TMS), where instance structural relationships, class-wise global dependencies, and the co-occurrence possibility of varying instance-label assignments are simultaneously taken into consideration in a unified and adaptive manner. Moreover, the parallelization capability of the Transformer enables efficient computation, making our model scalable to large-scale datasets. Specifically, we first represent instances and labels as nodes in the visual space and label space respectively, and then compute the hidden representation of each node in its individual space, by attending a self-attention strategy over its entire neighborhood. Subsequently, the cross-attention is adopted to excavate the correct assignments between instances and labels, and further interprets how classifying each label depends on the instances within an image and its interaction with other labels. Finally, an asymmetric focal loss is designed to optimize the instance-label correspondence, and read out image-level category confidences. Extensive experiments conducted on various multi-label image datasets demonstrate the superiority of our proposed method.
A Multihead Attention Self-Supervised Representation Model for Industrial Sensors Anomaly Detection

Industrial sensors capture critical information for intelligent manufacturing maintenance. To promote equipment upgrading and manufacturing processes, intelligent decisions, and information learning play an important role. Although deep learning methods historically obtain excellent results, there is always a trade-off between fine-tuning existing networks or designing models from scratch for sensor data processing. In this paper, we propose the multi-head attention self-supervised (MAS) representation model, which is a self-supervised learning-based sensor feature extraction network. To the best of our knowledge, this is the first time a self-supervised contrastive learning method using positive samples that represent multi-dimensional industry sensor data is being used for anomaly detection. We review alternative data augmentation methods proposed for better-representing sensor sequence data. We use this insight to design a new structure that adapts to the temporal characteristics of the application. We apply our method to a real-world water circulation system that uses a variety of industrial sensors. The effectiveness of the proposed MAS methods is demonstrated.
TrafAda: Cost-Aware Traffic Adaptation for Maximizing Bitrates in Live Streaming

The business growth of live streaming causes expensive bandwidth costs from the Content Delivery Network service. It necessitates traffic adaptation, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , adapting video bitrates for cost-efficient bandwidth utilization, especially under the 95 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>th</i> </sup> percentile pricing. However, our data-driven investigations indicate the existing methods are hard to achieve bitrate-cost balance in a long month-level billing cycle due to dynamic traffic patterns. We propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TrafAda</i> , a learning-based cost-aware traffic adaptation method consisting of i) an ultra-long-term bandwidth demand forecasting model to learn complex bandwidth usage patterns, and ii) an imitation learning-based bitrate decision mechanism to optimize the ultra-long-term objective. We have implemented and deployed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TrafAda</i> on a large-scale live streaming system in China serving over one billion viewers from 388 cities. The results show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TrafAda</i> improves peak-hour bitrate, quality of experience (QoE), and watching time by 34.75%, 44.56%, and 10.68%, respectively, without extra bandwidth cost, which can be converted to a considerable value for a commercial system.
Toward Fast Personalized Semi-Supervised Federated Learning in Edge Networks: Algorithm Design and Theoretical Guarantee

Recent years have witnessed a huge demand for artificial intelligence and machine learning applications in wireless edge networks to assist individuals with real-time services. Federated learning (FL) has emerged as a suitable and appealing distributed learning paradigm to deploy these applications at the network edge. Despite the many successful efforts made to apply FL to wireless edge networks, the adopted algorithms mostly follow the same spirit as FedAvg, thereby heavily suffering from the practical challenges of label deficiency and device heterogeneity. These challenges not only decelerate the model training in FL but also downgrade the application performance. In this paper, we focus on the algorithm design and address these challenges by investigating the personalized semi-supervised FL problem and proposing an effective algorithm, named FedCPSL. In particular, the techniques of pseudo-labeling, and interpolation-based model personalization are judiciously combined to provide a new problem formulation for personalized semi-supervised FL. The proposed FedCPSL algorithm employs novel strategies, including adaptive client variance reduction, local momentum, and normalized global aggregation, to combat the challenge of device heterogeneity and boost algorithm convergence. The convergence property of FedCPSL is also thoroughly analyzed and shows that FedCPSL is resilient to both statistical and system heterogeneity, obtaining a sublinear convergence rate. Experimental results on image classification tasks are presented to demonstrate that the proposed approach outperforms its counterparts in terms of both convergence speed and application performance.
Light Field Salient Object Detection With Sparse Views via Complementary and Discriminative Interaction Network

4D light field data record the scene from multiple views, thus implicitly providing beneficial depth cue for salient object detection in challenging scenes. Existing light field salient object detection (LF SOD) methods usually use a large number of views to improve the detection accuracy. However, using so many views for LF SOD brings difficulties to its practical applications. Considering that adjacent views in a light field are actually with very similar contents, in this work, we propose defining a more efficient pattern of input views, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., key sparse views, and design a network to effectively explore the depth cue from sparse views for LF SOD. Specifically, we firstly introduce a low rank-based statistical analysis to the existing LF SOD datasets, which allows us to conclude a fixed yet universal pattern for our key sparse views, including the number and positions of views. These views maintain the sufficient depth cue, but greatly lower the number of views to be captured and processed, facilitating practical applications. Then, we propose an effective solution with a key Complementary and Discriminative Interaction Module (CDIM) for LF SOD from key sparse views, named CDINet. The CDINet follows a two-stream structure to extract the depth cue from the light field stream ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., sparse views) and the appearance cue from the RGB stream ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., center view), generating features and initial saliency maps for each stream. The CDIM is tailored for inter-stream interaction of both these features and saliency maps, using the depth cue to complement the missing salient regions in RGB stream and discriminate the background distraction, to enhance the final saliency map further. Extensive experiments on three LF multi-view datasets demonstrate that our CDINet not only outperforms the state-of-the-art 2D methods, but also achieves competitive performance as compared with the state-of-the-art 3D and 4D methods. The code and results of our method are available at https://github.com/GilbertRC/LFSOD-CDINet.
Dynamic Clustering and Power Control for Two-Tier Wireless Federated Learning

Federated learning (FL) has been recognized as a promising distributed learning paradigm to support intelligent applications at the wireless edge, where a global model is trained iteratively through the collaboration of the edge devices without sharing their data. However, due to the relatively large communication cost between the devices and parameter server (PS), direct computing based on the information from the devices may not be resource efficient. This paper studies the joint communication and learning design for the over-the-air computation (AirComp)-based two-tier wireless FL scheme, where the lead devices first collect the local gradients from their nearby subordinate devices, and then send the merged results to the PS for the second round of aggregation. We establish a convergence result for the proposed scheme and derive the upper bound on the optimality gap between the expected and optimal global loss values. Next, based on the device distance and data importance, we propose a hierarchical clustering method to build the two-tier structure. Then, with only the instantaneous channel state information (CSI), we formulate the optimality gap minimization problem and solve it by using an efficient alternating minimization method. Numerical results show that the proposed scheme outperforms the baseline ones.
HVMM: A Holistic Virtual Machine Management Strategy for Cloud Data Centers

Cloud computing has emerged as an infrastructure in the era of digital economy and has been widely applied in various fields. Virtual Machine(VM) management is the key mechanism in a Cloud Data Center(CDC). A typical VM management system is responsible for VM allocation and VM reallocation, and it is usually designed to optimize specific objectives, especially for the metrics of energy consumption, resource wastage, communication cost, and Service Level Agreement Violations (SLAV). However, it is greatly challenging to optimize these metrics at the same time, and most existing VM management strategies focus on optimizing part of the above four metrics. In this paper, we propose a Holistic Virtual Machine Management (HVMM) strategy to optimize the energy consumption, resource wastage, communication cost, and SLAV simultaneously. First, we define two parameters, the Compatibility and the Performance-to-Power Ratio (PPR), for VM allocation to optimize energy consumption and resource wastage. Then, we propose a reallocation approach based on spectral clustering that can handle dynamic traffic between VMs without a priori knowledge of the traffic between VMs, and it takes a slight expense of resource wastage and energy consumption to reduce communication cost between VMs and ensures low SLAV risk. To evaluate the performance of HVMM, we compared the proposed strategy with the state-of-the-art strategies in various experiments on real-world traces. Compared with the other strategies, the resource wastage of HVMM is reduced by 62%. Simultaneously, the communication cost is reduced by about 26%, the energy consumption is reduced by about 5%, and SLAV is much lower than that of the others.
An Adaptive Post-Processing Network With the Global-Local Aggregation for Semantic Segmentation

Current semantic segmentation methods mainly focus on modeling the context of the global image to obtain high-quality segmentation results. However, they ignore the role of local image patches, which contain complementary and effective context information. In this paper, we propose an adaptive post-processing network (APPNet) for semantic segmentation based on the predictions of current methods in the global image and local image patches. The key point of APPNet is the global-local aggregation module, which models the context between global predictions and local predictions to generate accurate pixel-wise representation. Furthermore, we develop an adaptive points replacement module to compensate for the lack of fine detail in global prediction and the overconfidence in local predictions. Our method can be readily integrated into existing segmentation methods (i.e., ConvNeXt, HRNet, ViT-Adapter) with little memory and without extra modification in current models. We empirically demonstrate our method brings performance improvements across diverse datasets (i.e., Cityscapes, ADE20K, PASCAL-Context, COCO-Stuff). The code and models will be publicly available at https://github.com/zhu-gl-ux/APPN.
Multi-Task Learning With Multi-Query Transformer for Dense Prediction

Previous multi-task dense prediction studies developed complex pipelines such as multi-modal distillations in multiple stages or searching for task relational contexts for each task. The core insight beyond these methods is to maximize the mutual effects of each task. Inspired by the recent query-based Transformers, we propose a simple pipeline named Multi-Query Transformer (MQTransformer) that is equipped with multiple queries from different tasks to facilitate the reasoning among multiple tasks and simplify the cross-task interaction pipeline. Instead of modeling the dense per-pixel context among different tasks, we seek a task-specific proxy to perform cross-task reasoning via multiple queries where each query encodes the task-related context. The MQTransformer is composed of three key components: shared encoder, cross-task query attention module and shared decoder. We first model each task with a task-relevant query. Then both the task-specific feature output by the feature extractor and the task-relevant query are fed into the shared encoder, thus encoding the task-relevant query from the task-specific feature. Secondly, we design a cross-task query attention module to reason the dependencies among multiple task-relevant queries; this enables the module to only focus on the query-level interaction. Finally, we use a shared decoder to gradually refine the image features with the reasoned query features from different tasks. Extensive experiment results on two dense prediction datasets (NYUD-v2 and PASCAL-Context) show that the proposed method is an effective approach and achieves state-of-the-art results. Code and models are available at https://github.com/yangyangxu0/MQTransformer.