Reviewerly/exHarmony huggingface dataset:
Class-Balanced Universal Perturbations for Adversarial Training

Universal attack generates image-agnostic perturbation called universal adversarial perturbation (UAP), which can be added to all samples in the data distribution to fool the classifier. However, a universal perturbation will likely mislead the classifier to identify most adversarial examples as the same label, resulting in the imbalance of attack strength between classes. In this paper, we propose class-balanced UAPs that enlarge the dispersion of the predicted labels for adversarial examples. To ensure attack strength and balance simultaneously, we design a novel diversity objective containing probability calibration and penalty regularizer, which fully considers the predicted label distribution between samples and the predicted probability distribution within samples. Furthermore, we apply class-balanced attacks in adversarial training to defend against universal perturbations since the class-balanced UAP provides diverse perturbation directions. We correspondingly reformulate adversarial training from the min-max optimization problem into a new two-stage framework. Experiments on several benchmark datasets demonstrate that the class-balanced attack achieves better performance than the universal attack, while adversarial training with class-balanced UAP achieves state-of-the-art results in clean accuracy and robustness to universal perturbations.
Feature Attribution Explanation to Detect Harmful Dataset Shift

Detecting whether a distribution shift has occurred in the dataset is a critical aspect when implementing machine learning models, as even a small shift in the data distribution may largely affect the performance of a machine learning model and thus cause the deployed model to fail. In this work, we focus on detecting harmful dataset shifts, i.e., shifts that are detrimental to the performance of the machine learning model. The existing methods usually detect whether there is a shift between two datasets according to the following framework: first carrying out dimensionality reduction on the datasets, then determining whether dataset shift exists according to the two-sample statistical test(s) on the reduced datasets. The knowledge contained in the model trained on the dataset is not utilized in the above described dataset shift detection framework. To address this, this paper proposes to take advantage of explainable artificial intelligence (XAI) techniques to exploit the knowledge in trained models when detecting harmful dataset shifts. Specifically, we employ the feature attribution explanation (FAE) method to capture the knowledge in the model and combine it with a widely-used two-sample test method, i.e., maximum mean difference (MMD), to detect harmful dataset shifts. The experimental results on more than twenty different shifts in three widely used image datasets demonstrate that the proposed method is more effective in identifying harmful dataset shifts than existing methods. Moreover, experiments on several different models show that the method is robust and effective over different models, i.e., its detection performance is not sensitive to the model used.
CoLRP: A Contrastive Learning Abstractive Text Summarization Method with ROUGE Penalty

Contrastive learning can reduce the impact of ex-posure bias associated with training using maximum likelihood estimation, which aims to pull together positive samples to increase the likelihood of high-quality summaries and push away irrelevant negative samples to reduce the likelihood of low-quality summaries. In contrastive learning-based text summarization methods, a standard method for selecting positive and negative samples is randomly selected within a batch. This method can lead to sampling bias to the extent that the consistency of the representation space is compromised. Therefore, we propose a new method to penalize false negatives based on ROUGE metric scores as weights to sample from the dynamic output of the model training process. The method calculates ROUGE metric scores for penalizing false negatives in real-time and can distinguish between positive and negative samples to ensure spatial consistency and alleviate exposure bias. Experimental results on XSum, CNN/DM, and Multi-News datasets show that our approach effectively improves the performance of the latest text summarization pre-training models.
Effects of Spectral Normalization in Multi-Agent Reinforcement Learning

A reliable critic is central to on-policy actor-critic learning. But it becomes challenging to learn a reliable critic in a multi-agent sparse reward scenario due to two factors: 1) The joint action space grows exponentially with the number of agents 2) This, combined with the reward sparseness and environment noise, leads to large sample requirements for accurate learning. We show that regularising the critic with spectral normalization (SN) enables it to learn more robustly, even in multi-agent on-policy sparse reward scenarios. Our experiments show that the regularised critic is quickly able to learn from the sparse rewarding experience in the complex SMAC and RWARE domains. These findings highlight the importance of regularisation in the critic for stable learning.
McXai: Local Model-Agnostic Explanation As Two Games

To this day, various approaches for providing local explanation of black box machine learning models have been introduced. Despite these efforts, existing methods suffer from deficiencies such as being difficult to comprehend, only considering one feature at a time and disregarding inter-feature dependencies, lacking meaningful values for each feature, or only highlighting features that support the model's decision. To overcome these drawbacks, this study presents a new approach to explain the predictions of any black box classifier, called Monte Carlo tree search for eXplainable Artificial Intelligence (McXai). It employs a reinforcement learning strategy and models the explanation generation as two distinct games. In the first game, the objective is to identify feature sets that support the model's decision, while in the second game, the aim is to find feature sets that lead to alternative decisions. The output is a human-friendly representation in the form of a tree structure, where each node represents a set of features to be examined, with less specific interpretations at the top of the tree. Our experiments demonstrate that the features identified by McXai are more insightful with regard to the classifications compared to traditional algorithm like LIME and Gram-cam. Furthermore, the ability to identify misleading features provides guidance towards improved robustness of the black box classifier.
SS-Faster-RCNN: A Domain Adaptation-based Method to Detect Whether People Wear Masks Correctly

Since the outbreak of coronavirus (COVID-19) in 2019, wearing masks has been widely considered an effective method of reducing the risk of infection among people. However, incorrectly wearing a mask can significantly increase the risk of spreading the virus. To enable machines to automatically detect whether people are wearing masks correctly, we propose scenario-specific Faster-RCNN (SS-Faster-RCNN), a domain adaptation-based method for masked face detection. The frame-work is based on the Faster-RCNN and consists of two parts. The first part detects mask-wearing zones, and the last part aims to validate real mask faces in the candidate regions. The experimental results demonstrate that a trained feature extractor on a large mask face dataset can effectively enhance the model's performance on smaller mask face datasets for scenario-specific and non-scenario-specific cases. In addition, we also present videos for mask detection that consist of 858 seconds of video with 30 frames per second. The dataset mainly consists of images and videos from streets and subway stations. Overall, our method shows superior performance compared to others across different datasets. Codes, data, and evaluations are available at https://github.com/boranyang-ML/SSMFVD.
Cd-HRNN: Content-Driven HRNN to Improve Session-Based Recommendation System

The increasing popularity of digital entertainment systems has made personalization a key factor for success in the industry. Recommendation systems, particularly for videos and movies, are crucial in this regard. However, many existing systems are implicit feedback recommendation system that uses indirect signals to infer user preferences, such as user actions (e.g. clicks, views, purchases) or interactions with items (e.g. listening to a song, watching a movie). The challenge lies in the limited information and uncertainty present in user behavior, making it difficult to predict their interests and preferences. In Previous research, Recurrent Neural Networks (RNNs) have shown to be efficient in predicting the next item in a session, based on past item click sequences, but their effectiveness is limited when only relying on click sequences as input data. In this paper, we extend the Hierarchical RNN architecture (HRNN) for generating recommendations by combining session clicks and item content information, such as item ids and item description respectively. The Bidirectional Encoder Representations from Transformers (BERT) architecture is applied for generating feature vectors from text descriptions of the items. Our model has been extensively tested on the benchmark dataset MovieLens 1m has demonstrated superiority over state-of-the-art (SOTA) session-based recommendation systems (SBRS) models. Experimental results establish the efficacy of using content information along with item ids for recommendation.
Entropy Transformer Networks: A Learning Approach via Tangent Bundle Data Manifold

This paper focuses on an accurate and fast interpolation approach for image transformation employed in the design of CNN architectures. Standard Spatial Transformer Networks (STNs) use bilinear or linear interpolation as their interpolation, with unrealistic assumptions about the underlying data distributions, which leads to poor performance under scale variations. Moreover, STNs do not preserve the norm of gradients in propagation due to their dependency on sparse neighboring pixels. To address this problem, a novel Entropy STN (ESTN) is proposed that interpolates on the data manifold distributions. In particular, random samples are generated for each pixel in association with the tangent space of the data manifold, and construct a linear approximation of their intensity values with an entropy regularizer to compute the transformer parameters. A simple yet effective technique is also proposed to normalize the non-zero values of the convolution operation, to fine-tune the layers for gradients' norm-regularization during training. Experiments on challenging benchmarks show that the proposed ESTN can improve predictive accuracy over a range of computer vision tasks, including image reconstruction, and classification, while reducing the computational cost.
Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis

In this paper, we present a novel Liquid State Machine (LSM) based approach for modelling of multimodal longitudinal data: the Mosaic LSM. Our model harnesses the strengths of multiple LSMs, each designed to capture the temporal patterns of a specific data modality. This temporal information is then added to the raw data to create a composite representation that encompasses both the multimodal and the longitudinal aspects of the data. We demonstrate the performance of our approach on a real-world dataset that contains clinical, cognitive, and genetic modalities with the aim of predicting the Ultra-High Risk (UHR) status in individuals, six months in advance. Our results show that the Mosaic LSM outperforms traditional machine learning models, achieving an outstanding Matthew's Correlation Coefficient of 0.84 and prediction accuracy of 92.4%. Overall, our work highlights the potential of Mosaic LSM as a powerful tool for disease prognosis, and its ability to leverage both the multimodality and temporality of the data to improve performance.
Empowering Pandemic Response with Federated Learning for Protein Sequence Data Analysis

Genomics sequencing has become more accessible thanks to advances in genomics technology. We have witnessed this, particularly in the COVID-19 pandemic with massive growth in viral (SARS-CoV-2) protein sequence data generation. Massive data, however, has been underutilized as a result of certain pandemic policies and countermeasures. For political or economic reasons, some wary countries have purposely created this issue, rather than being subject to the natural barriers to the availability of these data. All countries cannot be expected to actively contribute fair data to the scientific community; otherwise, their participation may become passive. We require a strategy to encourage nations across the globe to fairly exchange information on the pandemic situation and data. We propose a federated learning (FL)-based model to address the issue of data privacy and enable real-time surveillance of epidemics. In FL, we train a feed-forward neural network locally and transmit the updated weights to the central server to combine the learning from local data. In this way, a federated learning-based architecture offers data security because there is no need to share the data (the data does not leave the premises; just parameters or weights are communicated) and encourages all countries to contribute fairly to the research. While FL is popular in dealing with image data, in this paper we apply it to bioinformatics, in particular, protein sequence classification. The results show that the FL-based model not only performs better than the centralized, traditional deep learning models (such as CNN, GRU, LSTM, and feed-forward neural network) in terms of predictive accuracy and fairness in data utilization but that all of this can be accomplished while maintaining data privacy. Our findings imply that FL can address the problems of data availability and privacy without compromising results. Employing factual data and tracking the evolution and dissemination of new SARS-CoV-2 lineages in real-time, can aid in the disclosure of verifiable advancements in pandemic studies.
Manually Crafted Chinese Text Corpus for Text Emotion Recognition

Aiming at the scarcity of high-quality emotion corpus resources in the current Chinese text emotion recognition task, we have manually completed the annotation of 35,000 text data in seven typical emotion classes. At the same time, we also proposed a text emotion recognition model ERNIE-BiLSTM based on enhanced language representation with information entities (ERNIE) and bidirectional long short-term memory network (BiLSTM). The model can effectively extract text semantics and capture context information, which can greatly improve the capability of text emotion recognition. Finally, the model achieved good recognition performance on our text emotion corpus, with the accuracy, precision, recall, and F1 score reaching as high as 93.29%, 93.05%, 93.04%, and 92.98%, respectively.
A RoBERTa-GlobalPointer-Based Method for Named Entity Recognition of Legal Documents

In legal practice, judicial professionals often need to extract useful information from numerous legal documents. The key to legal information extraction is the Named Entity Recognition (NER) of legal documents. To address three critical issues in NER of legal documents, this paper proposes a RoBERTa-GlobalPointer-based method for NER of legal documents. Specifically, we first use RoBERTa (a variant of the pre-trained language model BERT) to extract char-level feature representations of a legal document, and use the Skip-Gram method to extract its word-level feature representations, and fuse them to better capture the contextual information of entities in the document. Then, according to the concatenated result, we use the GlobalPointer method to calculate the score of each subsequence of the document, to which it is an entity of a certain type. Finally, we employ the balanced softmax function to determine whether or not a subsequence of the document is an entity of a certain type according to its score calculated by GlobalPointer. Our evaluation experiments on the Chinese judicial domain dataset show that the proposed method outperforms the state-of-the-art baseline methods.
X-MLP: A Patch Embedding-Free MLP Architecture for Vision

Convolutional neural networks (CNNs) and vision transformers (ViT) have obtained great achievements in computer vision. Recently, the research of multi-layer perceptron (MLP) architectures for vision have been popular again. Vision MLPs are designed to be independent from convolutions and self-attention operations. However, existing vision MLP architectures always depend on convolution for patch embedding. Thus we propose X-MLP, an architecture constructed absolutely upon fully connected layers and free from patch embedding. It decouples the features extremely and utilizes MLPs to interact the information across the dimension of width, height and channel independently and alternately. X-MLP is tested on ten benchmark datasets, all obtaining better performance than other vision MLP models. It even surpasses CNNs by a clear margin on various dataset. Furthermore, through mathematically restoring the spatial weights, we visualize the information communication between any couples of pixels in the feature map and observe the phenomenon of capturing long-range dependency.
Optimal Excitatory and Inhibitory Balance for High Learning Performance in Spiking Neural Networks with Long-Tailed Synaptic Weight Distributions

Excitatory/inhibitory (E/I) balance is significantly associated with cognitive function. Its imbalance impairs cognitive function, particularly in patients with psychiatric disorders. Recent physiological and modeling findings show that excitatory postsynaptic potentials (EPSPs) have a long-tailed distribution and contribute to the generation of spontaneous activity. Moreover, this spontaneous activity and its response to the external stimulus significantly alternate under the different E/I balance. However, the effects of the E/I balance under long-tailed EPSPs at the functional level remain unknown. Hence, to elucidate this relationship, we constructed a reservoir computing (RC) model generating the long-tailed distribution of EPSPs and investigated the effect of the E/I balance on the learning performance of RC in the memory capacity (MC) task, which measures how correctly delayed input signals can be reproduced. The results revealed that an appropriate E/I balance maximized the MC. This high MC was realized by recurrent spike propagation under long-tailed EPSPs. These findings contribute to the understanding of the effect of the E/I balance in physiologically relevant neural networks.
Quantum Convolutional Neural Network Architecture for Multi-Class Classification

We propose quantum circuit architectures for convolutional neural networks based on generalized 3-qubit and 2-qubit quantum gates for the multiclass classification problem. The quantum architecture is equivalent to a classical convolutional neural network with fully connected layers and densely connected layers. The quantum circuit parameters are optimized by minimizing the cross-entropy loss function. We validate the classification performance over several model configurations on the MNIST, Fashion-MNIST and Kuzushiji-MNIST datasets. Our proposed architecture shows classification accuracies that are comparable to classical CNNs with a similar number of parameters. In addition to this, we find that circuit depth is greatly decreased by a logarithmic factor compared to classical CNNs. We study the performance and complexity tradeoffs over several model configurations within the proposed quantum CNN architecture.
Improving Robustness Against Adversarial Attacks with Deeply Quantized Neural Networks

Reducing the memory footprint of Machine Learning (ML) models, particularly Deep Neural Networks (DNNs), is essential to enable their deployment into resource-constrained tiny devices. However, a disadvantage of DNN models is their vulnerability to adversarial attacks, as they can be fooled by adding slight perturbations to the inputs. Therefore, the challenge is how to create accurate, robust, and tiny DNN models deployable on resource-constrained embedded devices. This paper reports the results of devising a tiny DNN model, robust to adversarial black and white box attacks, trained with an automatic quantization-aware training framework, i.e. QKeras, with deep quantization loss accounted in the learning loop, thereby making the designed DNNs more accurate for deployment on tiny devices. We investigated how QKeras and an adversarial robustness technique, Jacobian Regularization (JR), can provide a co-optimization strategy by exploiting the DNN topology and the per layer JR approach to produce robust yet tiny deeply quantized DNN models. As a result, a new DNN model implementing this co-optimization strategy was conceived, developed and tested on three datasets containing both images and audio inputs, as well as compared its performance with existing benchmarks against various white-box and black-box attacks. Experimental results demonstrated that on average our proposed DNN model resulted in 8.3% and 79.5% higher accuracy than MLCommons/Tiny benchmarks in the presence of white-box and black-box attacks on the CIFAR-10 image dataset and a subset of the Google Speech Commands audio dataset respectively. It was also 6.5% more accurate in the presence of black-box attacks on the SVHN image dataset.
Prototype-Based Support Example Miner and Triplet Loss for Deep Metric Learning

Deep metric learning aims to learn a mapping function that projects input data into a high-dimensional embedding space, facilitating the clustering of similar data points while ensuring dissimilar ones are far apart. The most recent studies focus on designing a batch sampler and mining online triplets to achieve this purpose. Conventionally, hard negative mining schemes serve as the preferred batch sampler. However, most hard negative mining schemes search for hard examples in randomly selected mini-batches at each epoch, which often results in less-optimal hard examples and thus sub-optimal performances. Furthermore, Triplet Loss is commonly adopted to perform online triplet mining by pulling the hard positives close to and pushing the negatives away from the anchor. However, when the anchor in a triplet is an outlier, the positive example will be pulled away from the centroid of the cluster, thus resulting in a loose cluster and inferior performance. To address the above challenges, we propose the Prototype-based Support Example Miner (pSEM) and Triplet Loss (pTriplet Loss). First, we present a support example miner designed to mine the support classes on the prototype-based nearest neighbor graph of classes. Following this, we locate the support examples by searching for instances at the intersection between clusters of these support classes. Second, we develop a variant of Triplet Loss, referred to as a Prototype-based Triplet Loss. In our approach, a dynamically updated prototype is used to rectify outlier anchors, thus reducing their detrimental effects and facilitating a more robust formulation for Triplet Loss. Extensive experiments on typical Computer Vision (CV) and Natural Language Processing (NLP) tasks, namely person re-identification and few-shot relation extraction, demonstrated the effectiveness and generalizability of the proposed scheme, which consistently outperforms the state-of-the-art models.