Reviewerly/exHarmony huggingface dataset:
A noise-resistant graph neural network by semi-supervised contrastive learning

Authors:
raw_author_name: Zhengyu Lu
raw_affiliation_string: Guangxi Key Lab of Multi-source Information Ming & Security, Guangxi Normal University, Guilin, 541004, China; School of Computer Science and Information Engineering, Anyang Institute of Technology, Anyang, 455000, China
raw_author_name: Junbo Ma
raw_affiliation_string: School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, 310018, China
raw_author_name: Zongqian Wu
raw_affiliation_string: School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China
raw_author_name: Bo Zhou
raw_affiliation_string: Guangxi Collaborative Innovation Center of Modern Sericulture and Silk, Hechi University, Hechi, 546300, China
raw_author_name: Xiaofeng Zhu
raw_affiliation_string: Guangxi Key Lab of Multi-source Information Ming & Security, Guangxi Normal University, Guilin, 541004, China

Abstract: Graph neural networks (GNNs) have been widely applied for representation learning on the graph data in real applications, but few of them are designed to conduct representation learning on the graph data with noisy labels. Its key challenge is that the feature embeddings of nodes with noisy labels (noisy nodes for short) are close to those of unlabeled nodes so that the classifier constructed by GNNs is influenced by noisy nodes. To address this issue, in this paper, we propose a noise-resistant graph neural network with semi-supervised contrastive learning to push noisy nodes far away from unlabeled nodes in the embedding space. To do this, we design a constraint of semi-supervised contrastive learning and put it into the objective function of GNNs. Specifically, the proposed constraint enlarges the distance between noisy nodes and unlabeled nodes by pushing noisy nodes far away from their unlabeled neighbors in the embedding space. As a result, the embeddings of unlabeled nodes are influenced by noisy label less. Moreover, we intuitively analyze the feasibility of our proposed constraint. Comprehensive experiments on real datasets further verify the effectiveness of our proposed method over previous SOTA methods in terms of classification tasks with different ratio levels of noisy labels.
Triple Sequence Learning for Cross-domain Recommendation

Authors:
raw_author_name: Haokai Ma
raw_affiliation_string: School of Software, Shandong University, China
raw_author_name: Ruobing Xie
raw_affiliation_string: TEG, Tencent, China
raw_author_name: Lei Meng
raw_affiliation_string: School of Software, Shandong University; Shandong Research Institute of Industrial Technology, China
raw_author_name: Xin Chen
raw_affiliation_string: WeChat, Tencent, China
raw_author_name: Xu Zhang
raw_affiliation_string: WeChat, Tencent, China
raw_author_name: Leyu Lin
raw_affiliation_string: WeChat, Tencent, China
raw_author_name: Jie Zhou
raw_affiliation_string: WeChat, Tencent, China

Abstract: Cross-domain recommendation (CDR) aims at leveraging the correlation of users’ behaviors in both the source and target domains to improve the user preference modeling in the target domain. Conventional CDR methods typically explore the dual-relations between the source and target domains’ behaviors. However, this may ignore the informative mixed behaviors that naturally reflect the user’s global preference. To address this issue, we present a novel framework, termed triple sequence learning for cross-domain recommendation (Tri-CDR), which jointly models the source, target, and mixed behavior sequences to highlight the global and target preference and precisely model the triple correlation in CDR. Specifically, Tri-CDR independently models the hidden representations for the triple behavior sequences and proposes a triple cross-domain attention (TCA) method to emphasize the informative knowledge related to both user’s global and target-domain preference. To comprehensively explore the cross-domain correlations, we design a triple contrastive learning (TCL) strategy that simultaneously considers the coarse-grained similarities and fine-grained distinctions among the triple sequences, ensuring the alignment while preserving information diversity in multi-domain. We conduct extensive experiments and analyses on six cross-domain settings. The significant improvements of Tri-CDR with different sequential encoders verify its effectiveness and universality. The source code is available at https://github.com/hulkima/Tri-CDR .
Multi-operator opposition-based learning with the neighborhood structure for numerical optimization problems and its applications

Authors:
raw_author_name: Jiahang Li
raw_affiliation_string: State Key Lab of Intelligent Manufacturing Equipment & Technology, Huazhong University of Science & Technology, Wuhan, 430074, PR China
raw_author_name: Liang Gao
raw_affiliation_string: State Key Lab of Intelligent Manufacturing Equipment & Technology, Huazhong University of Science & Technology, Wuhan, 430074, PR China
raw_author_name: Xinyu Li
raw_affiliation_string: State Key Lab of Intelligent Manufacturing Equipment & Technology, Huazhong University of Science & Technology, Wuhan, 430074, PR China

Abstract: Opposition-based learning (OBL) is an effective strategy that adjusts the population to accelerate the convergence of the algorithm. However, OBL involves two phases (generation and calculation) that are seldom mentioned simultaneously. Besides, individual information in OBL is not fully used to guide evolution. First, a novel neighborhood generation strategy is proposed to fully use individual information. There are numerous variants of the neighborhood structure. Three prominent neighborhood structures are designed to demonstrate their benefits in various dimensions. Second, a multi-operator calculation method with two distributions is introduced to reach the equilibrium between diversification and intensification. Consequently, this paper proposes three multi-operator neighborhood-based OBL variants (MNOBLs). Compared with the traditional OBL variants, MNOBLs generate the opposite population by using a neighborhood of the individuals. A well-balanced calculation method can lead to good convergence behavior. Subsequently, the proposed MNOBLs are embedded into the basic differential evolution (DE) algorithm to obtain the proposed algorithms. The extensive and rigorous numerical experiments are designed at three levels: strategies, algorithms, and functions. Specifically, the effectiveness and efficiency of the proposed algorithms are verified by comparing them with 13 state-of-the-art DE variants and four metaheuristics on the CEC 2017, and CEC 2022 test suites, and two real-world optimization problems. The comprehensive results show that the proposed OBL variants provide promising results for most problems.
Explainability for Large Language Models: A Survey

Authors:
raw_author_name: Haiyan Zhao
raw_affiliation_string: New Jersey Institute of Technology, Newark, USA
raw_author_name: Hanjie Chen
raw_affiliation_string: Johns Hopkins University, Baltimore, USA
raw_author_name: Fan Yang
raw_affiliation_string: Wake Forest University, Winston-Salem, USA
raw_author_name: Ninghao Liu
raw_affiliation_string: University of Georgia, Athens, USA
raw_author_name: Huiqi Deng
raw_affiliation_string: Shanghai Jiao Tong University, Shanghai, China
raw_author_name: Hengyi Cai
raw_affiliation_string: Institute of Computing Technology, CAS, China
raw_author_name: Shuaiqiang Wang
raw_affiliation_string: Baidu Inc., Beijing, China
raw_author_name: Dawei Yin
raw_affiliation_string: Baidu Inc., Beijing, China
raw_author_name: Mengnan Du
raw_affiliation_string: New Jersey Institute of Technology, Newark, USA

Abstract: Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transparency poses unwanted risks for downstream applications. Therefore, understanding and explaining these models is crucial for elucidating their behaviors, limitations, and social impacts. In this article, we introduce a taxonomy of explainability techniques and provide a structured overview of methods for explaining Transformer-based language models. We categorize techniques based on the training paradigms of LLMs: traditional fine-tuning-based paradigm and prompting-based paradigm. For each paradigm, we summarize the goals and dominant approaches for generating local explanations of individual predictions and global explanations of overall model knowledge. We also discuss metrics for evaluating generated explanations and discuss how explanations can be leveraged to debug models and improve performance. Lastly, we examine key challenges and emerging opportunities for explanation techniques in the era of LLMs in comparison to conventional deep learning models.
Balancing Innovation and Security in the Cloud

Authors:
raw_author_name: S. Boopathi
raw_affiliation_string: Mechanical Engineering, Muthayammal Engineering College (Autonomous), India

Abstract: This chapter delves into the intricate relationship between innovation and security in the digital age. It highlights the challenges of the digital age and the transformative impact of cloud computing, emphasizing its role in driving innovation. The chapter also delves into cloud service and deployment models, discussing the benefits and challenges of cloud security. It also discusses the role of innovation in driving progress through case studies and addressing challenges organizations face. The chapter also discusses risk assessment and mitigation in the cloud, compliance, regulatory challenges, legal and ethical considerations, and best practices. It also explores emerging technologies like AI and machine learning, Zero Trust security, and future directions.
Autoencoders and their applications in machine learning: a survey

Authors:
raw_author_name: Kamal Berahmand
raw_affiliation_string: School of Computer Science, Faculty of Science, Queensland University of Technology (QUT), Brisbane, Australia
raw_author_name: Fatemeh Daneshfar
raw_affiliation_string: Department of Computer Engineering, University of Kurdistan, Sanandaj, Iran
raw_author_name: Yuefeng Li
raw_affiliation_string: School of Computer Science, Faculty of Science, Queensland University of Technology (QUT), Brisbane, Australia
raw_author_name: Yue Xu
raw_affiliation_string: School of Computer Science, Faculty of Science, Queensland University of Technology (QUT), Brisbane, Australia

Abstract: Abstract Autoencoders have become a hot researched topic in unsupervised learning due to their ability to learn data features and act as a dimensionality reduction method. With rapid evolution of autoencoder methods, there has yet to be a complete study that provides a full autoencoders roadmap for both stimulating technical improvements and orienting research newbies to autoencoders. In this paper, we present a comprehensive survey of autoencoders, starting with an explanation of the principle of conventional autoencoder and their primary development process. We then provide a taxonomy of autoencoders based on their structures and principles and thoroughly analyze and discuss the related models. Furthermore, we review the applications of autoencoders in various fields, including machine vision, natural language processing, complex network, recommender system, speech process, anomaly detection, and others. Lastly, we summarize the limitations of current autoencoder algorithms and discuss the future directions of the field.
Variational quantum time evolution without the quantum geometric tensor

Authors:
raw_author_name: Julien Gacon
raw_affiliation_string: IBM Quantum, IBM Research Europe – Zurich, CH-8803 Rüschlikon, Switzerland; Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
raw_author_name: Jannes Nys
raw_affiliation_string: Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
raw_author_name: Stefan Woerner
raw_affiliation_string: IBM Quantum, IBM Research Europe – Zurich, CH-8803 Rüschlikon, Switzerland
raw_author_name: Giuseppe Carleo
raw_affiliation_string: Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland

Abstract: Real- and imaginary-time quantum state evolutions are crucial in physics and chemistry for exploring quantum dynamics, preparing ground states, and computing thermodynamic observables. On near-term devices, variational quantum time evolution is a promising candidate for these tasks, as the required circuit model can be tailored to the available devices' capabilities. Due to the evaluation of the quantum geometric tensor (QGT), however, this approach quickly becomes infeasible for relevant system sizes. Here, we propose a dual formulation for variational time evolution, which replaces the calculation of the QGT by solving a fidelity-based optimization to compute updates to the dynamics in each time step. We demonstrate our algorithm for the time evolution of the Heisenberg Hamiltonian and show that it accurately reproduces the system dynamics at a fraction of the cost of standard variational quantum time evolution algorithms.
Analysis on unit maximum capacity of orthogonal multiple watermarking for multimedia signals in B5G wireless communications

Authors:
raw_author_name: Mianjie Li
raw_affiliation_string: Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, 510006, China; School of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou, 510665, China
raw_author_name: Jiao Wang
raw_affiliation_string: School of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou, 510665, China
raw_author_name: Zhihong Tian
raw_affiliation_string: Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, 510006, China
raw_author_name: Nadra Guizani
raw_affiliation_string: School of Electrical Engineering and Computer Science, Washington State University, Pullman, WA, 99164, USA
raw_author_name: Xiaojiang Du
raw_affiliation_string: Department of Computer and Information Science, Temple University, Philadelphia, PA, 19122, USA
raw_author_name: Chun Shan
raw_affiliation_string: School of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou, 510665, China

Abstract: B5G aims to meet the demands of today's growing mobile traffic and expand the communication space. Considering that the intelligent application to B5G wireless communications will involve the security issues of user data and operational data, in this paper, the maximum capacity problem of the multi-watermarking method in multimedia signal hiding is studied to alleviate the information security problem of B5G. In the analyzed multi-watermarking process, the Spread Transform Dither Modulation is employed. During the watermarking procedures, through Gram-Schmidt orthogonalization, we can obtain the multiple spreading vectors. Consequently, the multiple watermarks can be simultaneously embedded into the same position of multimedia signal. Moreover, the multiple watermarks can be extracted without affecting one another during the extraction process. In our research, we intensively analyze the effect of the size of the spreading vector on the unit maximum capacity; consequently, we discover the hidden theoretical relationship between the size of the spreading vector and the unit maximum capacity. Many experiments have been conducted to seek out the optimal parameter values, with which the method can achieve the highest robustness, on the premise of possessing high capacity and good imperceptibility.
Transformer Driven Matching Selection Mechanism for Multi-Label Image Classification

Authors:
raw_author_name: Yanan Wu
raw_affiliation_string: Key Laboratory of Big Data and Artificial Intelligence in Transportation, Ministry of Education, and the School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
raw_author_name: Songhe Feng
raw_affiliation_string: Key Laboratory of Big Data and Artificial Intelligence in Transportation, Ministry of Education, and the School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
raw_author_name: Gongpei Zhao
raw_affiliation_string: Key Laboratory of Big Data and Artificial Intelligence in Transportation, Ministry of Education, and the School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
raw_author_name: Yi Jin
raw_affiliation_string: Key Laboratory of Big Data and Artificial Intelligence in Transportation, Ministry of Education, and the School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China

Abstract: Graph Matching has recently emerged as an attractive technique applied to various computer vision tasks. Graph Matching based multi-label image classification, in particular, treats each image as a bag of instances and reformulates the classification task as an instance-label matching selection problem, achieving state-of-the-art results on diverse benchmarks. However, the generalization and scalability of such learned model cannot be well guaranteed due to its manually predetermined graph structure and high-dimension embedding of dense connections between instances and labels. To address these limitations, in this work, we propose a novel Transformer Driven Matching Selection framework for Multi-Label Image Classification (C-TMS), where instance structural relationships, class-wise global dependencies, and the co-occurrence possibility of varying instance-label assignments are simultaneously taken into consideration in a unified and adaptive manner. Moreover, the parallelization capability of the Transformer enables efficient computation, making our model scalable to large-scale datasets. Specifically, we first represent instances and labels as nodes in the visual space and label space respectively, and then compute the hidden representation of each node in its individual space, by attending a self-attention strategy over its entire neighborhood. Subsequently, the cross-attention is adopted to excavate the correct assignments between instances and labels, and further interprets how classifying each label depends on the instances within an image and its interaction with other labels. Finally, an asymmetric focal loss is designed to optimize the instance-label correspondence, and read out image-level category confidences. Extensive experiments conducted on various multi-label image datasets demonstrate the superiority of our proposed method.
A Multihead Attention Self-Supervised Representation Model for Industrial Sensors Anomaly Detection

Authors:
raw_author_name: Jinhu Lü
raw_affiliation_string: School of Automation Science and Electrical Engineering, State Key Laboratory of Software Development Environment, Beihang University, Beijing, China
raw_author_name: Tian Wang
raw_affiliation_string: Institute of Artificial Intelligence, the State Key Laboratory of Software Development Environment, Beihang University, Beijing, China
raw_author_name: Kexin Liu
raw_affiliation_string: School of Automation Science and Electrical Engineering, State Key Laboratory of Software Development Environment, Beihang University, Beijing, China
raw_author_name: Baochang Zhang
raw_affiliation_string: Institute of Artificial Intelligence, the State Key Laboratory of Software Development Environment, Beihang University, Beijing, China
raw_author_name: Hichem Snoussi
raw_affiliation_string: Charles Delaunay Institute (ICD), University of Technology of Troyes, Troyes, France

Abstract: Industrial sensors capture critical information for intelligent manufacturing maintenance. To promote equipment upgrading and manufacturing processes, intelligent decisions, and information learning play an important role. Although deep learning methods historically obtain excellent results, there is always a trade-off between fine-tuning existing networks or designing models from scratch for sensor data processing. In this paper, we propose the multi-head attention self-supervised (MAS) representation model, which is a self-supervised learning-based sensor feature extraction network. To the best of our knowledge, this is the first time a self-supervised contrastive learning method using positive samples that represent multi-dimensional industry sensor data is being used for anomaly detection. We review alternative data augmentation methods proposed for better-representing sensor sequence data. We use this insight to design a new structure that adapts to the temporal characteristics of the application. We apply our method to a real-world water circulation system that uses a variety of industrial sensors. The effectiveness of the proposed MAS methods is demonstrated.
Toward Fast Personalized Semi-Supervised Federated Learning in Edge Networks: Algorithm Design and Theoretical Guarantee

Authors:
raw_author_name: Shuai Wang
raw_affiliation_string: Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Tampines, Singapore
raw_author_name: Yanqing Xu
raw_affiliation_string: School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China
raw_author_name: Yanli Yuan
raw_affiliation_string: School of Cyberspace Science and Technology, Beijing Institute of Technology, Beijing, China
raw_author_name: Tony Q. S. Quek
raw_affiliation_string: Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Tampines, Singapore

Abstract: Recent years have witnessed a huge demand for artificial intelligence and machine learning applications in wireless edge networks to assist individuals with real-time services. Federated learning (FL) has emerged as a suitable and appealing distributed learning paradigm to deploy these applications at the network edge. Despite the many successful efforts made to apply FL to wireless edge networks, the adopted algorithms mostly follow the same spirit as FedAvg, thereby heavily suffering from the practical challenges of label deficiency and device heterogeneity. These challenges not only decelerate the model training in FL but also downgrade the application performance. In this paper, we focus on the algorithm design and address these challenges by investigating the personalized semi-supervised FL problem and proposing an effective algorithm, named FedCPSL. In particular, the techniques of pseudo-labeling, and interpolation-based model personalization are judiciously combined to provide a new problem formulation for personalized semi-supervised FL. The proposed FedCPSL algorithm employs novel strategies, including adaptive client variance reduction, local momentum, and normalized global aggregation, to combat the challenge of device heterogeneity and boost algorithm convergence. The convergence property of FedCPSL is also thoroughly analyzed and shows that FedCPSL is resilient to both statistical and system heterogeneity, obtaining a sublinear convergence rate. Experimental results on image classification tasks are presented to demonstrate that the proposed approach outperforms its counterparts in terms of both convergence speed and application performance.