Reviewerly/exHarmony huggingface dataset:
Learning Superior Cooperative Policy in Adversarial Multi-Team Reinforcement Learning

Multi-agent Reinforcement Learning (MARL) has become a powerful tool for addressing multi-agent challenges. Existing studies have explored numerous models to use MARL to solve single-team cooperation (competition) problems and adversarial problems with opponents controlled by static knowledge-based policies. However, most studies in the literature often ignore adversarial multi-team problems involving dynamically evolving opponents. We investigate adversarial multi-team problems where all participating teams use MARL learners to learn policies against each other. Two objectives are achieved in this study. Firstly, we design an adversarial team-versus-team learning framework to generate cooperative multi-agent policies to compete against opponents without preprogrammed opponent partners or any supervision. Secondly, we explore the key factors to achieve win-rate superiority during dynamic competitions. Then we put forward a novel FeedBack MARL (FBMARL) algorithm that takes advantage of feedback loops to adjust optimizer hyper-parameters based on real-time game statistics. Finally, the effectiveness of our FBMARL model is tested in a benchmark environment named Multi-Team Decentralized Collective Assault (MT-DCA). The results demonstrate that our feedback MARL model can achieve superior performance over baseline competitor MARL learners in 2-team and 3-team dynamic competitions.
Towards Inference of Original Graph Data Information from Graph Embeddings

This paper studies to what extent an adversary (without the original graph data) can recover the original raw graph data from graph embeddings. To quantify the original graph data information leakage from graph embeddings, we develop a deep neural network model InferNet that can be used by adversaries to infer the original graph data information from an adversary-accessible graph embedding database. Specifically, we propose the data-free reversed knowledge distillation (KD) technique to support InferNet training even if the original graph dataset is absent. To improve the performance of InferNet, we design two cycle-consistency loss functions to have an interactive training of InferNet over three series of datasets. Our intensive experiments demonstrate that InferNet can infer the original graph data information from the graph embedding dataset with high accuracy.
Universal Recurrent Event Memories for Streaming Data

In this paper, we propose a new event memory architecture (MemNet) for recurrent neural networks, which is universal for different types of time series data such as scalar, multivariate or symbolic. Unlike other external neural memory architectures, it stores key-value pairs, which separate the information for addressing and for content to improve the representation, as in the digital archetype. Moreover, the key-value pairs also avoid the compromise between memory depth and resolution that applies to memories constructed by the model state. One of the MemNet key characteristics is that it requires only linear adaptive mapping functions while implementing a nonlinear operation on the input data. MemNet architecture can be applied without modifications to scalar time series, logic operators on strings, and also to natural language processing, providing state-of-the-art results in all application domains such as the chaotic time series, the symbolic operation tasks, and the question-answering tasks (bAbI). Finally, controlled by five linear layers, MemNet requires a much smaller number of training parameters than other external memory networks as well as the transformer network. The space complexity of MemNet equals a single self-attention layer. It greatly improves the efficiency of the attention mechanism and opens the door for IoT applications.
Improving Knowledge Tracing with Diverse Question Factors

Knowledge Tracing (KT) aims at predicting future students’ performance based on their responses to historical questions, which plays a pivotal role in intelligent education. Most of the existing efforts pay more attention to students’ dynamically changing knowledge states while neglecting the individualization of questions and difficulty level that differs from question to question. Towards this end, in this paper, we propose a novel Diverse Question Factors-enhanced Knowledge Tracing (DQFKT) method, which sufficiently explores various question factors to generate better prediction. On one hand, two reliable and low-dimensional student-concept connection spaces are established through exploiting the student-question-concept relations and individualization of question, and then fine-grained response representations of questions are obtained according to students’ responses. On the other hand, difficulty level with concrete concept for each question is introduced to simulate the complex interactions between students and questions for improving the prediction performance. Extensive experiments on three datasets have demonstrated that the DQFKT approach has more precise prediction of student performance and stronger interpretable power.
A Parameterized Cubic Bézier Spline-based Informed RRT* for Non-holonomic Path Planning

This paper proposes a new path planning algorithm for robotics called Informed SRRT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> . Compared to conventional RRT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> algorithms with Euclidean metrics, our algorithm extends the approach by incorporating a local planner from SRRT to satisfy both external and internal constraints. To compute the path to the goal region, we use parameterized cubic curves instead of computationally expensive numerical methods. We add two extra lines at the endpoints of the Bézier spline to leave rooms for the rewiring process. Kinematic constraints require at least three state connections to be tweaked during rewiring. The algorithm always ensures that the path has G <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> continuity of curvature within upper-bound constraints. Simulation results demonstrate that the proposed method finds shorter paths than SRRT while maintaining the same iteration of node sampling.
SMKD: Selective Mutual Knowledge Distillation

Mutual knowledge distillation (MKD) is a technique used to transfer knowledge between multiple models in a collaborative manner. However, it is important to note that not all knowledge is accurate or reliable, particularly under challenging conditions such as label noise, which can lead to models that memorize undesired information. This problem can be addressed by improving the reliability of the knowledge source, as well as selectively selecting reliable knowledge for distillation. While making a model more reliable is a widely studied topic, selective MKD has received less attention. To address this, we propose a new framework called selective mutual knowledge distillation (SMKD). The key component of SMKD is a generic knowledge selection formulation, which allows for either static or progressive selection thresholds. Additionally, SMKD covers two special cases: using no knowledge and using all knowledge, resulting in a unified MKD framework. We present extensive experimental results to demonstrate the effectiveness of SMKD and justify its design.
Snapshot Spectral Clustering - a costless approach to deep clustering ensembles generation

Despite tremendous advancements in Artificial Intelligence, learning from large sets of data in an unsupervised manner remains a significant challenge. Classical clustering algorithms often fail to discover complex dependencies in large datasets, especially considering sparse, high-dimensional spaces. However, deep learning techniques proved to be successful when dealing with large quantities of data, efficiently reducing their dimensionality without losing track of underlying information. Several interesting advancements have already been made to combine deep learning and clustering. Still, the idea of enhancing the clustering results by combining multiple views of the data generated by deep neural networks appears to be insufficiently explored yet. This paper aims to investigate this direction and bridge the gap between deep neural networks, clustering techniques and ensemble learning methods. To achieve this goal, we propose a novel deep clustering ensemble method - Snapshot Spectral Clustering, designed to maximize the gain from combining multiple data views while minimizing the computational costs of creating the ensemble. Comparative analysis and experiments described in this paper prove the proposed concept, while the conducted hyperparameter study provides a valuable intuition to follow when selecting proper values.
SwiftTron: An Efficient Hardware Accelerator for Quantized Transformers

Transformers' compute- intensive operations pose enormous challenges for their deployment in resource- constrained EdgeAI / tiny ML devices. As an established neural network compression technique, quantization reduces the hardware computational and memory resources. In particular, fixed-point quantization is desirable to ease the computations using lightweight blocks, like adders and multipliers, of the underlying hardware. However, deploying fully-quantized Transformers on existing general-purpose hardware, generic AI accelerators, or specialized architectures for Transformers with floating-point units might be infeasible and/or inefficient. Towards this, we propose SwiftTron, an efficient specialized hardware accelerator designed for Quantized Transformers. SwiftTron supports the execution of different types of Transformers' operations (like Attention, Softmax, GELU, and Layer Normalization) and accounts for diverse scaling factors to perform correct computations. We synthesize the complete SwiftTron architecture in a 65 nm CMOS technology with the ASIC design flow. Our Accelerator executes the RoBERTa-base model in 1.83 ns, while consuming 33.64 mW power, and occupying an area of 273 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> • To ease the reproducibility, the RTL of our SwiftTron architecture is released at https://github.com/albertomarchisio/SwiftTron.
Data Under Siege: The Quest for the Optimal Convolutional Autoencoder in Side-Channel Attacks

Encryption is a method to keep our data safe from third parties. However, side-channel information may be leaked during encryption due to physical properties. This information can be used in side-channel attacks to recover critical values such as the secret encryption key. To this end, it is necessary to understand the robustness of implementations to assess the security of data handled by a device. Side-channel attacks are one such method which allow researchers to evaluate the robustness of implementations using appropriate metrics. In the security community, machine learning is playing a prominent role in the study of side-channel attacks. A notable example of this is the use of Convolutional Autoencoders (CAE) as a preprocessing step on the measurements. In this work we study in depth the problem of finding the most suitable architecture of such Convolutional Autoencoders. To this end, Optuna is used to explore the CAE hyperparameter space. This process allows us to identify hyperparameters that outperform state-of-the-art autoencoders, reducing the needed traces for a succesful attack by approximately 37 % in the presence of Gaussian noise and reducing the trainable parameters needed to attack desynchronization by a factor of 29. In addition to the promising results, experiments carried out in this paper allow a better understanding of the hyperparameter space in the field of side channel attacks, providing a solid base for future use of CAE in this specific domain.
Two-stage Neural Architecture Optimization with Separated Training and Search

Neural architecture search (NAS) has been a popular research topic for designing deep neural networks (DNNs) automatically. It is able to improve the design efficiency of neural architectures significantly for given learning tasks. Recently, instead of conducting architecture search in the original neural architecture space, many NAS approaches have been proposed to learn continuous representations from neural architectures for architecture search or estimation. In particular, Neural Architecture Optimization (NAO) is a representative method which encodes neural architectures as continuous representations by an auto-encoder and then performs continuous optimization in the encoded space with gradient-based methods. However, as NAO only considers the top-ranked architectures in learning the continuous representation, it could fail to construct a satisfied continuous optimization space which contains the expected high-quality neural architectures. Taking this cue, in this paper we propose a two-stage NAO (TNAO) to learn a more completed continuous representation of neural architectures which could provide a better optimization space for NAS. Specifically, by designing a pipeline that separates the training and search stages, we first build the training set via random sampling from the entire neural architecture search space, which is with the aim of collecting the well-distributed neural architectures for training. Moreover, to exploit the architectural semantic information with limited data effectively, we propose an improved Transformer auto-encoder for learning the continuous representation, which is supervised by ranking information of the neural architecture performance. Lastly, towards more effective optimization of neural architectures, we adopt a population-based swarm intelligence algorithm, i.e. competitive swarm optimization (CSO), with a newly designed remapping scoring scheme. To evaluate the efficiency of the proposed TNAO, comprehensive experimental studies are conducted on two common search spaces, i.e., NAS-Bench-101 and NAS-Bench-201. The architecture with the top 0.02% performance is discovered on NAS-Bench-101 and the best architecture in the CIFAR-10 dataset is obtained on NAS-Bench-201.
LisaCLIP: Locally Incremental Semantics Adaptation towards Zero-shot Text-driven Image Synthesis

The automatic transfer of a plain photo into a desired synthetic style has attracted numerous users in the application fields of photo editing, visual art, and entertainment. By connecting images and texts, the Contrastive Language- Image Pre-Training (CLIP) model facilitates the text-driven style transfer without exploring the image's latent domain. However, the trade-off between content fidelity and stylization remains challenging. In this paper, we present LisaCLIP, a CLIP- based image synthesis framework that only exploits the CLIP model to guide the imagery manipulations with a depth-adaptive encoder-decoder network. Since an image patch's semantics depend on its size, LisaCLIP progressively downsizes the patches while adaptively selecting the most significant ones for further stylization. We introduce a multi-stage training strategy to speed up LisaCLIP's convergence by decoupling the optimization objectives. Various experiments on public datasets demonstrated that LisaCLIP supported a wide range of style transfer tasks and outperformed other state-of-the-art methods in maintaining the balance between content and style.
Improving Relation Extraction by Entity-Level Contrastive Learning

Relation Extraction (RE) is a fundamental task in Natural Language Processing (NLP), which aims to extract relations between entity pairs mentioned in a given sentence. Recently, relation extraction has achieved remarkable process with the development of deep neural network. However, most of the recent works focus on using external knowledge to pretrain model or encoding entity related information with graph neural network. In this paper, we propose a BERT-based model with entity-level contrastive learning module based only on the original input sentences for relation extraction. By optimizing the contrastive learning and relation classification objectives jointly, our model is more effective in encoding relation representation between two entities in a given sentence and achieves better relation extraction performance. Experiments conducted on two mainstream RE datasets show that our model significantly outperforms the baselines' results on two benchmark datasets. Meanwhile, the results of our model are very competitive with the state-of-the-art models which employ extra training datas or information.
Evolving Constrained Reinforcement Learning Policy

Evolutionary algorithms have been used to evolve a population of actors to generate diverse experiences for training reinforcement learning agents, which helps to tackle the temporal credit assignment problem and improves the exploration efficiency. However, when adapting this approach to address constrained problems, balancing the trade-off between the reward and constraint violation is hard. In this paper, we propose a novel evolutionary constrained reinforcement learning (ECRL) algorithm, which adaptively balances the reward and constraint violation with stochastic ranking, and at the same time, restricts the policy's behaviour by maintaining a set of Lagrange relaxation coefficients with a constraint buffer. Extensive experiments on robotic control benchmarks show that our ECRL achieves outstanding performance compared to state-of-the-art algorithms. Ablation analysis shows the benefits of introducing stochastic ranking and constraint buffer.
Accommodating Self-attentional Heterophily Topology into High- and Low-pass Graph Convolutional Network for Skeleton-based Action Recognition

In the scene of human skeleton-based action recognition, graph convolutional network (GCN) is widely applied to model human action dynamics and achieves extraordinary results, where graph topology is in charge of feature extraction and plays a crucial role in learning representation in GCNs. This paper presents a novel high- and low-pass graph convolutional network (HLP-GCN), aiming to effectively aggregate joint features based on learned homophily and heterophily topologies using self-attention modules for skeleton-based action recognition tasks. Specifically, such an HLP-GCN framework first learns homophily and heterophily topologies dynamically so as to capture different correlations between input samples and then combine similar information arising from homophily topologies with dissimilar ones from heterophily topologies to get the final embeddings. Furthermore, a temporal modeling module is added to this HLPGCN framework, and a new pooling strategy called global node (GNode) is cascaded subsequently, in order to have the extraction of more robust spatiotemporal features for skeleton action recognition. Finally, the experimental results demonstrate that our method outperforms previous state-of-the-art methods by 0.2%, 0.4%, and 0.3% on three large-scale action recognition datasets including NTU RGB+D X-Set, NTU RGB+D 120 X-Sub, and NW-UCLA, respectively.
Incorporating Least-Effort Loss to Stabilize Training of Wasserstein GAN

In order to further improve the convergence properties of generative adversarial networks, in this paper, we analyze how the stability can be affected by the so-called best-effort manner of the discriminator in the minimax game. We point out that this manner can cause the multistate problem and the optimization entangling problem. To alleviate these, we proposed an alternative least-effort loss to regularize the training behaviors of the discriminator. With this loss, the discriminator only updates when it is unable to distinguish distributions. To evaluate the effectiveness of the least-effort loss, we introduce it into Wasserstein GAN. Experiments on Dirac delta distribution and image datasets demonstrate that the least-effort loss can effectively improve the convergence properties and generation quality of WGAN. Furthermore, the behaviors of the discriminator and generator during the training show that, with the least-effort loss, the state space of the discriminator shrinks, and the optimization of the discriminator and the generator disentangles in some way.
HSimCSE: Improving Contrastive Learning of Unsupervised Sentence Representation with Adversarial Hard Positives and Dual Hard Negatives

Recently, contrastive learning (CL) has emerged as the fundamental framework for learning better sentence representations. In the unsupervised sentence representation task, due to the lack of labeled data, current CL-based approaches generally use various methods to generate or select positive and negative samples for the given sentence. Despite their success, existing CL-based unsupervised sentence representation methods underestimate hard positive samples and hard negative samples, which do not fully exploit the power of contrastive learning. In this paper, we argue that we need to focus more on hard positive and hard negative samples. To this end, we propose a novel contrastive learning model, HSimCSE, that extends SimCSE by considering both the hard positive and hard negative samples. Specifically, we first propose a novel adversarial positive sample generation module to generate an adversarial hard positive sample, then we propose a dual negative sample selection module to select hard negative samples from the in-batch samples and the entire training corpus. Finally, we propose a quadruplet loss to minimize the distance between the anchor sample and the adversarial hard positive sample and maximize the distance between the anchor sample and the two hard negative samples. Experiments conducted on seven semantic text similarity tasks demonstrate the effectiveness of our method. The source code can be found at https://github.com/xubodhu/HSimCSE.
Event-Guided Attention Network for Low Light Image Enhancement

In the low-light conditions, images are corrupted by low contrast and severe noise, but event cameras can capture event streams with clear edge structures. Therefore, we propose an Event-Guided Attention Network for Low-light Image Enhancement (EGAN) using a dual branch Network and recover clear structure with the guide of events. To overcome the lack of paired training datasets, we first synthesize the dataset containing low-light event streams, low-light images, and the ground truth (GT) normal-light images. Then, we develop an end-to-end dual branch network consisting of a Image Enhancement Branch (IEB) and a Gradient Reconstruction Branch (GRB). The GRB branch reconstructs image gradients using events, and the IEB enhances low-light images using reconstructed gradients. Moreover, we develops the Attention based Event-Image Feature Fusion Module (AEIFFM) which selectively fuses the event and low-light image features using the spatial and channel attention mechanism, and the fused features are concatenated into the IEB and GRB, which respectively generate the enhanced images with clear structure and more accurate gradient images. Extensive experiments on synthetic and real datasets demonstrate that the proposed EGAN produces visually more appealing enhancement images, and achieves a good performance in structure preservation and denoising over state-of-the-arts.