Reviewerly/exHarmony huggingface dataset:
Temporal Inconsistency-Based Intrinsic Reward for Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) has shown promising results in many challenging sequential decision-making tasks. Recently, deep neural networks have dominated this field. However, the policy networks of agent's may fall into local optimum during the training phase, which severely constrains the performance of exploration. To address this issue, we propose a novel MARL learning framework named PSAM, which contains a new temporal inconsistency-based intrinsic reward and a diversity control strategy. Specifically, we save the parameters of the deep models along the optimization path of the agent's policy network, which can be denoted as snapshots. Through measuring the difference between snapshots, we can employ the difference as an intrinsic reward. Moreover, we propose a diversity control strategy to improve the performance further. Finally, to verify the effectiveness of the proposed method, we conduct extensive experiments in several widely used MARL environments. The results show that in many environments, PSAM can not only achieve state-of-the-art performance and prevent the policy network from getting stuck in local minima but also accelerate the agent's learning of the policy. It is worth noting that the proposed regularizer can be used using a plug-and-play manner without introducing any additional hyper-parameters and training costs.
AKD: Using Adversarial Knowledge Distillation to Achieve Black-box Attacks

Though Deep Neural Networks(DNNs) have achieved excellent performance in computer vision(CV) tasks such as classification, they are vulnerable to adversarial examples which are generated by adding small-magnitude perturbation to inputs. Recently, different methods have been proposed to produce adversarial examples, most of them are white-box attacks, which assume the adversary knows the internal structure and parameters of the target model which is not practical in the real world. So we proposed using adversarial knowledge distillation to iteratively train a substitute model for the target black model to better learn its high-frequency information processing for giving input and then generate adversarial examples based on the substitute model to attack the target model. According to the experiments, our method can achieve nearly or even a higher attacking success rate than directly attacking the target model in a white-box setting. Besides, most white-box attack methods can achieve black-box attacks using our approach.
Zero-Knowledge Attack for Replicating Protected Deep Neural Networks

As deep neural networks constantly improve and provide state-of-the-art solutions to various problems, deployment of these models becomes more common, and so does the importance of protecting these models against malicious attacks attempting to replicate these models. In this paper, we present a novel zero-knowledge method for attacking and stealing knowledge from deep neural networks. Our method utilizes unlabeled data and the predictions of the mentor model we would like to steal. The presented method targets the most protected models which reveal only the minimal amount of information, i.e., the predicted label. We assume no access to any internal information about the model, and no access to the training data. The presented method improves the SOTA performance of attacking protected neural network models. The results show that all classification neural networks are vulnerable to the presented attack method, and any attacker can effectively replicate these models without having access to their architecture, parameters, training data, or softmax outputs.
Automatic Parameter Optimisation Framework for ECoS-based Models

Over the past decade there has been efforts to address a major issue in obtaining high-performing Machine Learning (ML) models; that of more systematic methods to establish the optimal hyper-parameters for learning involving automated hyper-parameter selection which would increase the accuracy of these ML models. These methods have largely been driven by the popularity of the development of deep learning models, and traditional ML models have also benefited from these advances. One class of ML models where these hyper-parameter optimization techniques have rarely been applied is to a class of neural-network based learners called Evolving-Connectionist Systems (ECoS). To determine whether ECoS-based learners could improve their performance by benefitting from such hyper-parameter optimization methods is the focus of this research. In this paper we investigate how three different techniques for hyper-parameter optimization govern the learning of two variants of ECoS-based learners: The Self-Evolving Connectionist System (SECoS) and the Evolving Fuzzy Neural Network (EFuNN). Results of these experiments indicate that the Tree-structured Parzen Estimator hyper-parameter optimization algorithm works well for both SECoS and EFuNN learning on classification tasks. There are, however, unanswered questions as to the degree to which the efficacy of an appropriate hyper-parameter optimization framework can be adopted for ECoS-based learners.
An Answer FeedBack Network for Visual Question Answering

Recent advances have explored the power of transformer architecture in Visual Question Answering(VQA). However, most of the models suffer from misalignment of multimodal features, and they focus on unimportant image regions when answering the given questions. To address this, in this paper, we propose an Answer FeedBack Network (AFBN) to focus on image region features that are more beneficial for answering questions. The generate answers of the backbone network are again inputted into the network as feedback information. Then, we propose a FeedBack module (FB) to control the answer feedback. Additionally, we adopt the consistency loss function to reconstruct the image region features. By this function, the model can ensure the same of the image region features related to the question or answer. Extensive experiments on VQA-v2 benchmark dataset show that our method achieves better performance than the state-of-the-art methods.
Homeostatic Reinforcement Learning through Soft Behavior Switching with Internal Body State

The embodied autonomous agent, such as a house-hold robot or a pet robot, is required to satisfy multiple requirements simultaneously. One possible approach would be to train the agent to simultaneously solve multiple tasks using a single physical entity (i.e., the body). However, integrating multiple tasks is not a trivial problem in the reward design paradigm of reinforcement learning (RL). Homeostatic RL treats the integration of multiple tasks as a control problem over the agent's internal bodily state. Still, learning in previous studies was extremely slow. In this study, we report novel behavior switching architectures: the Interoceptive Mixture of Experts (IMoE) and the Interoceptive Behavior Switching (IBS) for homeostatic RL agents with continuous motor control. In these architectures, the agent switches between multiple policies using internal body states (interoception). We tested IMoE and IBS in four homeostatic RL environments. We also compared IMoE and IBS with a fully connected model and a query key value switching model with full observation policies. The results indicate that the proposed architectures provide better or competitive results in all four benchmark environments.
FraudAmmo: Large Scale Synthetic Transactional Dataset for Payment Fraud Detection

Global losses due to payment fraud have tripled from $9.84 billion in 2011 to $32.39 billion in 2020 and are expected to reach $40.62 billion by 2027. In addition to the financial losses, fraud negatively impacts brand reputation and leads to a bad customer experience. Advanced machine learning has been actively adopted to tackle the fraud detection problem at scale. However, the scarcity of open datasets leads to less reproducible research, especially in the payments domain. We have released a synthetic transactional dataset, FraudAmmo, containing 3 million transactions, synthetically generated from real-world datasets. FraudAmmo is diverse with respect to transactional channels, geography, customer, and fraud types. We leverage the idea of privacy preservation in tabular Generative Adversarial Neural Networks (GANs) to generate FraudAmmo. In addition to privacy-preserved GANs, we have applied certain checks to ensure the customer's differential privacy. The quality of the generated dataset is evaluated on various metrics including machine learning efficacy, statistical similarity, and privacy preservability. We have also benchmarked our results on FraudAmmo using machine learning algorithms such as bagging, boosting, MLP, and logistic regression. To the best of our knowledge, this is the first large-scale synthetic fraud transaction dataset that aims to aid academia and research groups in developing and validating their fraud detection models. The dataset can be found here - https://github.com/karthi2107/FraudAmmo
Generative Models vs Discriminative Models: Which Performs Better in Detecting Cyberbullying in Memes?

Accessibility to the internet has led to a massive increase in the usage of social networking and online communication apps over the past decade. With so much content available online, these platforms suffer from governance and monitoring problems making the users susceptible to online cyberbullying and trolling. Recently, this trolling and hate comes majorly from memes that combine text and image modalities. Many studies show how cyberbullying can harm the mental well-being of the affected individuals. Previous studies also tried to study the role of sentiment, emotions, and sarcasm in identifying hateful memes setting up the meme detection task as a multi-modal, multitask problem. In the past, meme detection models were discriminative, but more recently generative models have been used to solve non-generation tasks such as aspect-based sentiment analysis and span detection. Motivated by this, in this work, we propose a unified Multimodal Generative framework, MGex by reframing the multitasking problem of detecting cyberbullying, sentiment, emotion, and sarcasm as a multimodal text-to-text generation problem. For this purpose, we use MultiBully dataset which provides annotation for all these labels. Here, we evaluate and contrast our proposed generative framework with several multitasking baselines and state-of-the-art models. Disclaimer: The article contains offensive text and profanity. This is owing to the nature of the work and does not reflect any opinion or stand of the authors.
Improving the Transferability of Adversarial Attacks through Experienced Precise Nesterov Momentum

Deep neural networks are vulnerable to adversarial examples. Although the adversarial example has superior white-box attack success rate, its transferability is poor under the black-box setting. Momentum is often integrated into attacks so as to prevent adversarial examples from overfitting the source model and improve the transferability of adversarial examples. How-ever, conventional momentum merely accumulates few gradients during the early iterations, resulting in the early adversarial examples already overfitting the source model. Therefore, we propose Experienced Momentum (EM), which is trained on a set of models derived by Random Channels Swapping (RCS). Since EM takes the direction of loss increasing for multiple models into account, assigning EM to the initial value of momentum to makes adversarial examples transferable across models during the early iterations. Moreover, conventional Nesterov momentum only take the previous gradients into consideration but ignore the gradient of the current data point during the whole pre-update, making the estimate of the next position imprecise. It prompts us to propose Precise Nesterov momentum (PN), which not only retains the looking-ahead property but also adopts the gradient of the current data point during the pre-update. To further improve transferability, we combine EM and PN as Experienced Precise Nesterov momentum (EPN). Extensive experiments on the ImageNet dataset against normally trained and defense models demonstrate that the proposed EPN is more effective than conventional momentum for improving transferability.
A Hierarchical Vision Transformer Using Overlapping Patch and Self-Supervised Learning

Transformer-based network architectures have gradually replaced convolutional neural networks in computer vision. Compared with convolutional neural networks, Transformer is able to learn global information of images and has better feature extraction capability. However, due to the lack of inductive bias, vision Transformers require a large amount of data for pre-training, such as ViT. Local-based Transformers effectively reduce the computational complexity, but could not establish long-range dependencies and do not perform as well on small-scale datasets. In response to these problems, OPSe Transformer is proposed. A global attention calculation module is designed to be added behind each stage of the vision Transformer, using a slightly larger and overlapping key patch and value patch to enhance the exchange of information between two adjacent windows and to aggregate global information in the local Transformer. In addition, a self-supervised learning proxy task is added to the architecture, corresponding to the loss function of the proxy task to constrain the training of the model on the dataset, so that the vision Transformer can learn spatial information within an image and improve the training effect of the network. Comparative experiments are conducted on the tiny-ImageNet, CIFAR-10/100, and other datasets, and the experimental results show that compared with the baseline algorithm, our model improves the accuracy by up to 3.91%.
WokeGPT: Improving Counterspeech Generation Against Online Hate Speech by Intelligently Augmenting Datasets Using a Novel Metric

With hate speech spreading rapidly online, it is increasingly important to respond automatically. However, there are some critical limitations in developing systems which produce these responses, which are known as counterspeeches. First, datasets containing paired instances of a hate speech and its appropriate response are very small. There is an abundance of hate speech on the web and in structured datasets, but quality counterspeeches are rare. Thus, since data is scarce, there is a need for automated methods to intelligently increase the size of existing paired datasets. Another critical challenge is that existing Natural Language Generation (NLG) metrics are not suitable for evaluating such systems, because these metrics do not accurately reflect how a human interprets the relationship between a hate speech and its counterspeech. Lastly, language models trained on internet text often exhibit a large amount of bias, which is unsuitable for sensitive tasks such as counterspeech generation. To address these challenges, we first introduce a technique to intelligently augment a small paired dataset of hate speech and counterspeech to make it substantially larger and varied, through a pairing technique that appropriately matches unpaired instances of hate speech with synthetic and existing counterspeeches. Next, we identified a need for a metric that evaluates counterspeech in the same way humans do, and propose a novel metric called PD-Score that leverages an advanced debating system. We empirically show through a large survey, that existing NLG metrics correlate poorly to human assessment and that our alternative is much more tightly bound to human assessment. Lastly, we curated a large domain-specific text corpus called WokeCorpus which we use to pretrain the language model before finetuning it for producing counterspeeches. We show that this both debiases the language model and aids performance.
How AI Affects Our Understanding of Musical Works That Should Be Protected by Copyright

Applying Artificial Intelligence (AI) algorithms in creative works challenges the justifications and effectiveness of Copyright Protection aimed at human authors. AI -assisted musical works influence the value of rights held by stakeholders in the music industry associated with human artists' earnings, the production of new original works, and public access to pro-tected compositions. The lenient approach that allows computer-generated works to invoke copyright protection questions how the law perceives what constitutes ‘original’ works. The philosophical research question of this paper asks, ‘how do AI-assisted works affect our understanding of musical creation and their copyright protection.’ The work incorporates a doctrinal approach to expose how AI tools compromise the presumptions behind the originality criteria of creative freedom, independent creation, and human authors' labor, skill, and judgment. Following the creative process adopted in compositions and exposing elements that attract copyright protection, the discourse reveals how AI imposes a quantifiable nature upon melodies. The findings show that the limited spectrum for musical creation and the expedited and low-cost process of AI-assisted compositions undermine justifications behind copyright protection. The paper seeks to expose and bridge the gap between AI tools application and their socio-legal impact on the music industry.
Rethinking the Trigger-injecting Position in Graph Backdoor Attack

Backdoor attacks have been demonstrated as a security threat for machine learning models. Traditional backdoor attacks intend to inject backdoor functionality into the model such that the backdoored model will perform abnormally on inputs with predefined backdoor triggers and still retain state-of-the-art performance on the clean inputs. While there are already some works on backdoor attacks on Graph Neural Networks (GNNs), the backdoor trigger in the graph domain is mostly injected into random positions of the sample. There is no work analyzing and explaining the backdoor attack performance when injecting triggers into the most important or least important area in the sample, which we refer to as trigger-injecting strategies MIAS and LIAS, respectively. Our results show that, generally, LIAS performs better, and the differences between the LIAS and MIAS performance can be significant. Furthermore, we explain these two strategies’ similar (better) attack performance through explanation techniques, which results in a further understanding of backdoor attacks in GNNs.
Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Study

The excellent performance of deep neural networks is usually accompanied by a large number of parameters and computations, which have limited their usage on the resource- limited edge devices. To address this issue, abundant methods such as pruning, quantization and knowledge distillation have been proposed to compress neural networks and achieved significant breakthroughs. However, most of these compression methods focus on the architecture or the training method of neural networks but ignore the influence from data augmentation. In this paper, we revisit the usage of data augmentation in model compression and give a comprehensive study on the relation between model sizes and their optimal data augmentation policy. To sum up, we mainly have the following three observations: (A) Models in different sizes prefer data augmentation with different magnitudes. Hence, in iterative pruning, data augmentation with varying magnitudes leads to better performance than data augmentation with a consistent magnitude. (B) Data augmentation with a high magnitude may significantly improve the performance of large models but harm the performance of small models. Fortunately, small models can still benefit from strong data augmentations by firstly learning them with “additional parameters” and then discard these “additional parameters” during inference. (C) The prediction of a pre-trained large model can be utilized to measure the difficulty of data augmentation. Thus it can be utilized as a criterion to design better data augmentation policies. We hope this paper may promote more research on the usage of data augmentation in model compression.
Weight Compander: A Simple Weight Reparameterization for Regularization

Regularization is a set of techniques that are used to improve the generalization ability of deep neural networks. In this paper, we introduce weight compander (WC), a novel effective method to improve generalization by reparameterizing each weight in deep neural networks using a nonlinear function. It is a general, intuitive, cheap and easy to implement method, which can be combined with various other regularization techniques. Large weights in deep neural networks are a sign of a more complex network that is overfitted to the training data. Moreover, regularized networks tend to have a greater range of weights around zero with fewer weights centered at zero. We introduce a weight reparameterization function which is applied to each weight and implicitly reduces overfitting by restricting the magnitude of the weights while forcing them away from zero at the same time. This leads to a more democratic decision-making in the network. Firstly, individual weights cannot have too much influence in the prediction process due to the restriction of their magnitude. Secondly, more weights are used in the prediction process, since they are forced away from zero during the training. This promotes the extraction of more features from the input data and increases the level of weight redundancy, which makes the network less sensitive to statistical differences between training and test data. From an optimizational point of view, the second effect of WC can be seen as a reactivation of “dead” (near zero) weights to participate in the training. This increases the probability to find an ensemble of weights which performs better in the given task. We extend our method to learn the hyperparameters of the introduced weight reparameterization function. This avoids hyperparameter search and gives the network the opportunity to align the weight reparameterization with the training progress. We show experimentally that using weight compander in addition to standard regularization methods improves the performance of neural networks. Furthermore, we empirically analyze the weight distribution with and without weight compander after training to confirm the companding effects of our method on the weights.