Reviewerly/exHarmony huggingface dataset:
Dual Adversarial Graph Neural Networks for Multi-label Cross-modal Retrieval

Cross-modal retrieval has become an active study field with the expanding scale of multimodal data. To date, most existing methods transform multimodal data into a common representation space where semantic similarities between items can be directly measured across different modalities. However, these methods typically suffer from following limitations: 1) They usually attempt to bridge the modality gap by designing losses in the common representation space which may not be sufficient to eliminate potential heterogeneity of different modalities in the common space. 2) They typically treat labels as independent individuals and ignore label relationships which are important for constructing semantic links between multimodal data. In this work, we propose a novel Dual Adversarial Graph Neural Networks (DAGNN) composed of the dual generative adversarial networks and the multi-hop graph neural networks, which learn modality-invariant and discriminative common representations for cross-modal retrieval. Firstly, we construct the dual generative adversarial networks to project multimodal data into a common representation space. Secondly, we leverage the multi-hop graph neural networks, in which a layer aggregation mechanism is proposed to exploit multi-hop propagation information, to capture the label correlation dependency and learn inter-dependent classifiers. Comprehensive experiments conducted on two cross-modal retrieval benchmark datasets, NUS-WIDE and MIRFlickr, indicate the superiority of DAGNN.
Emotionally Informed Hate Speech Detection: A Multi-target Perspective

Hate Speech and harassment are widespread in online communication, due to users' freedom and anonymity and the lack of regulation provided by social media platforms. Hate speech is topically focused (misogyny, sexism, racism, xenophobia, homophobia, etc.), and each specific manifestation of hate speech targets different vulnerable groups based on characteristics such as gender (misogyny, sexism), ethnicity, race, religion (xenophobia, racism, Islamophobia), sexual orientation (homophobia), and so on. Most automatic hate speech detection approaches cast the problem into a binary classification task without addressing either the topical focus or the target-oriented nature of hate speech. In this paper, we propose to tackle, for the first time, hate speech detection from a multi-target perspective. We leverage manually annotated datasets, to investigate the problem of transferring knowledge from different datasets with different topical focuses and targets. Our contribution is threefold: (1) we explore the ability of hate speech detection models to capture common properties from topic-generic datasets and transfer this knowledge to recognize specific manifestations of hate speech; (2) we experiment with the development of models to detect both topics (racism, xenophobia, sexism, misogyny) and hate speech targets, going beyond standard binary classification, to investigate how to detect hate speech at a finer level of granularity and how to transfer knowledge across different topics and targets; and (3) we study the impact of affective knowledge encoded in sentic computing resources (SenticNet, EmoSenticNet) and in semantically structured hate lexicons (HurtLex) in determining specific manifestations of hate speech. We experimented with different neural models including multitask approaches. Our study shows that: (1) training a model on a combination of several (training sets from several) topic-specific datasets is more effective than training a model on a topic-generic dataset; (2) the multi-task approach outperforms a single-task model when detecting both the hatefulness of a tweet and its topical focus in the context of a multi-label classification approach; and (3) the models incorporating EmoSenticNet emotions, the first level emotions of SenticNet, a blend of SenticNet and EmoSenticNet emotions or affective features based on Hurtlex, obtained the best results. Our results demonstrate that multi-target hate speech detection from existing datasets is feasible, which is a first step towards hate speech detection for a specific topic/target when dedicated annotated data are missing. Moreover, we prove that domain-independent affective knowledge, injected into our models, helps finer-grained hate speech detection.
Deduplicating Training Data Makes Language Models Better

We find that existing language modeling datasets contain many near-duplicate examples and long repetitive substrings.As a result, over 1% of the unprompted output of language models trained on these datasets is copied verbatim from the training data.We develop two tools that allow us to deduplicate training datasets-for example removing from C4 a single 61 word English sentence that is repeated over 60,000 times.Deduplication allows us to train models that emit memorized text ten times less frequently and require fewer training steps to achieve the same or better accuracy.We can also reduce train-test overlap, which affects over 4% of the validation set of standard datasets, thus allowing for more accurate evaluation.Code for deduplication is released at https://github.com/googe-research/ dedup icate-text-datasets.
Self-Adversarial Training Incorporating Forgery Attention for Image Forgery Localization

Image editing techniques enable people to modify the content of an image without leaving visual traces and thus may cause serious security risks. Hence the detection and localization of these forgeries become quite necessary and challenging. Furthermore, unlike other tasks with extensive data, there is usually a lack of annotated forged images for training due to annotation difficulties. In this paper, we propose a self-adversarial training strategy and a reliable coarse-to-fine network that utilizes a self-attention mechanism to localize forged regions in forgery images. The self-attention module is based on a Channel-Wise High Pass Filter block (CW-HPF). CW-HPF leverages inter-channel relationships of features and extracts noise features by high pass filters. Based on the CW-HPF, a self-attention mechanism, called <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>forgery attention</i></b> , is proposed to capture rich contextual dependencies of intrinsic inconsistency extracted from tampered regions. Specifically, we append two types of attention modules on top of CW-HPF respectively to model internal interdependencies in spatial dimension and external dependencies among channels. We exploit a coarse-to-fine network to enhance the noise inconsistency between original and tampered regions. More importantly, to address the issue of insufficient training data, we design a self-adversarial training strategy that expands training data dynamically to achieve more robust performance. Specifically, in each training iteration, we perform adversarial attacks against our network to generate adversarial examples and train our model on them. The proposed method is based on the assumption of content-changed manipulations. Extensive experimental results demonstrate that our proposed algorithm steadily outperforms state-of-the-art methods by a clear margin in different benchmark datasets.
Deep active learning for object detection

Active learning (AL) for object detection (OD) aims to reduce labeling costs by selecting the most valuable samples that enhance the detection network from the unlabeled pool. Due to the complexity of OD compared with image classification, more consideration should be given when designing the selection strategies. Previous works have studied aggregating information of multiple outputs (especially the location information) and aggregating information of batch boxes, all of which indicate improved performances. However, the evaluation index—mean average precision (mAP) has not been considered seriously, although improving it is the goal of AL. Moreover, the background class is far more than other classes (15:1 or more) in each batch of samples, leading to a class imbalance problem. Therefore, AL strategies for OD, which take mAP and class imbalance in batch into consideration, may perform better. In this paper, WBetGS is proposed, which not only considers aggregating information of multiple outputs and batch boxes but also aims to mAP improvement and to address the class imbalance in batch. A weighted algorithm is introduced to promote the mAP more effectively. Besides, WBetGS eliminates the impact of class imbalance between background and object categories by extracting class-balanced information. Moreover, a diversity and uncertainty based sampling algorithm is introduced for batch mode active learning in object detection. The experimental results demonstrate that our method performs better than basic methods, saving up 100% of the labeling efforts while reaching the same performance in an actual industrial application.
Bibliometric Using Vosviewer with Publish or Perish (using Google Scholar data): From Step-by-step Processing for Users to the Practical Examples in the Analysis of Digital Learning Articles in Pre and Post Covid-19 Pandemic

The purpose of this study is to analyze and demonstrate step-by-step bibliometric data analysis using VOSViewer completely and systematically. Step-by-step analysis was provided to make first-time users easily following the way how to use VOSviewers. This report allows and provides an easy way of data analysis by utilizing mapping tools and provide analysis of research developments regarding digital learning media. The method used in this research is to perform bibliometric analysis to produce network visualization of co-work maps and density maps of co-works. The analysis was carried out with the number of publications obtained, relating to the predetermined topics totaling 88 documents in 2017-2021. As practical examples, we evaluated the analysis of digital learning articles in pre and post covid. We found that VOSviewer can be used to give suggestions in the data analysis results.
Entangled two-photon absorption by atoms and molecules: A quantum optics tutorial

Two-photon absorption (TPA) and other nonlinear interactions of molecules with time–frequency-entangled photon pairs have been predicted to display a variety of fascinating effects. Therefore, their potential use in practical quantum-enhanced molecular spectroscopy requires close examination. This Tutorial presents a detailed theoretical study of one- and two-photon absorption by molecules, focusing on how to treat the quantum nature of light. We review some basic quantum optics theory and then we review the density-matrix (Liouville) derivation of molecular optical response, emphasizing how to incorporate quantum states of light into the treatment. For illustration, we treat in detail the TPA of photon pairs created by spontaneous parametric down conversion, with an emphasis on how quantum light TPA differs from that with classical light. In particular, we treat the question of how much enhancement of the TPA rate can be achieved using entangled states. This Tutorial includes a review of known theoretical methods and results as well as some extensions, especially the comparison of TPA processes that occur via far-off-resonant intermediate states only and those that involve off-resonant intermediate states by virtue of dephasing processes. A brief discussion of the main challenges facing experimental studies of entangled two-photon absorption is also given.
The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders

Multi-task learning with transformer encoders (MTL) has emerged as a powerful technique to improve performance on closely-related tasks for both accuracy and efficiency while a question still remains whether or not it would perform as well on tasks that are distinct in nature.We first present MTL results on five NLP tasks, POS, NER, DEP, CON, and SRL, and depict its deficiency over single-task learning.We then conduct an extensive pruning analysis to show that a certain set of attention heads get claimed by most tasks during MTL, who interfere with one another to fine-tune those heads for their own objectives.Based on this finding, we propose the Stem Cell Hypothesis to reveal the existence of attention heads naturally talented for many tasks that cannot be jointly trained to create adequate embeddings for all of those tasks.Finally, we design novel parameter-free probes to justify our hypothesis and demonstrate how attention heads are transformed across the five tasks during MTL through label analysis.Performance % of Attention Heads Kept PS/S STL STL-SP STL-DP MTL-DP STL-SP STL-DP MTL-DP STL STL-DP
A truncated SVD-based ARIMA model for multiple QoS prediction in mobile edge computing

In the mobile edge computing environments, Quality of Service (QoS) prediction plays a crucial role in web service recommendation.Because of distinct features of mobile edge computing, i.e., the mobility of users and incomplete historical QoS data, traditional QoS prediction approaches may obtain less accurate results in the mobile edge computing environments.In this paper, we treat the historical QoS values at different time slots as a temporal sequence of QoS matrices.By incorporating the compressed matrices extracted from QoS matrices through truncated Singular Value Decomposition (SVD) with the classical ARIMA model, we extend the ARIMA model to predict multiple QoS values simultaneously and efficiently.Experimental results show that our proposed approach outperforms the other state-of-the-art approaches in accuracy and efficiency.
Fine-Tuning Wav2Vec2 for Speaker Recognition

This paper explores applying the wav2vec2 framework to speaker recognition instead of speech recognition. We study the effectiveness of the pre-trained weights on the speaker recognition task, and how to pool the wav2vec2 output sequence into a fixed-length speaker embedding. To adapt the framework to speaker recognition, we propose a single-utterance classification variant with CE or AAM softmax loss, and an utterance-pair classification variant with BCE loss. Our best performing variant, w2v2-aam, achieves a 1.88% EER on the extended voxceleb1 test set compared to 1.69% EER with an ECAPA-TDNN baseline. Code is available at https://github.com/nikvaessen/w2v2-speaker.
Enhance to read better: A Multi-Task Adversarial Network for Handwritten Document Image Enhancement

Handwritten document images can be highly affected by degradation for different reasons: Paper ageing, daily-life scenarios (wrinkles, dust, etc.), bad scanning process and so on. These artifacts raise many readability issues for current Handwritten Text Recognition (HTR) algorithms and severely devalue their efficiency. In this paper, we propose an end to end architecture based on Generative Adversarial Networks (GANs) to recover the degraded documents into a clean and readable form. Unlike the most well-known document binarization methods, which try to improve the visual quality of the degraded document, the proposed architecture integrates a handwritten text recognizer that promotes the generated document image to be more readable. To the best of our knowledge, this is the first work to use the text information while binarizing handwritten documents. Extensive experiments conducted on degraded Arabic and Latin handwritten documents demonstrate the usefulness of integrating the recognizer within the GAN architecture, which improves both the visual quality and the readability of the degraded document images. Moreover, we outperform the state of the art in H-DIBCO challenges, after fine tuning our pre-trained model with synthetically degraded Latin handwritten images, on this task.
MetaICL: Learning to Learn In Context

We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks.This metatraining enables the model to more effectively learn a new task in context at test time, by simply conditioning on a few training examples with no parameter updates or task-specific templates.We experiment on a large, diverse collection of tasks consisting of 142 NLP datasets including classification, question answering, natural language inference, paraphrase detection and more, across seven different metatraining/target splits.MetaICL outperforms a range of baselines including in-context learning without meta-training and multi-task learning followed by zero-shot transfer.We find that the gains are particularly significant for target tasks that have domain shifts from the meta-training tasks, and that using a diverse set of the meta-training tasks is key to improvements.We also show that MetaICL approaches (and sometimes beats) the performance of models fully finetuned on the target task, and outperforms much bigger models with nearly 8x parameters.Finally, we show that MetaICL is complementary to human-written instructions, and the best performance can be achieved by combining both approaches.
Past, Present, and Future: Conversational Emotion Recognition through Structural Modeling of Psychological Knowledge

Conversational Emotion Recognition (CER) is a task to predict the emotion of an utterance in the context of a conversation.Although modeling the conversational context and interactions between speakers has been studied broadly, it is important to consider the speaker's psychological state, which controls the action and intention of the speaker.The state-of-the-art method introduces CommonSense Knowledge (CSK) to model psychological states in a sequential way (forwards and backwards).However, it ignores the structural psychological interactions between utterances.In this paper, we propose a pSychological-Knowledge-Aware Interaction Graph (SKAIG).In the locally connected graph, the targeted utterance will be enhanced with the information of action inferred from the past context and intention implied by the future context.The utterance is self-connected to consider the present effect from itself.Furthermore, we utilize CSK to enrich edges with knowledge representations and process the SKAIG with a graph transformer.Our method achieves state-of-theart and competitive performance on four popular CER datasets.
Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection From Point Clouds

Previous works for LiDAR-based 3D object detection mainly focus on the single-frame paradigm. In this paper, we propose to detect 3D objects by exploiting temporal information in multiple frames, i.e., point cloud videos. We empirically categorize the temporal information into short-term and long-term patterns. To encode the short-term data, we present a Grid Message Passing Network (GMPNet), which considers each grid (i.e., the grouped points) as a node and constructs a k-NN graph with the neighbor grids. To update features for a grid, GMPNet iteratively collects information from its neighbors, thus mining the motion cues in grids from nearby frames. To further aggregate long-term frames, we propose an Attentive Spatiotemporal Transformer GRU (AST-GRU), which contains a Spatial Transformer Attention (STA) module and a Temporal Transformer Attention (TTA) module. STA and TTA enhance the vanilla GRU to focus on small objects and better align moving objects. Our overall framework supports both online and offline video object detection in point clouds. We implement our algorithm based on prevalent anchor-based and anchor-free detectors. Evaluation results on the challenging nuScenes benchmark show superior performance of our method, achieving first on the leaderboard (at the time of paper submission) without any "bells and whistles." Our source code is available at https://github.com/shenjianbing/GMP3D.
Empirical performance bounds for quantum approximate optimization

The quantum approximate optimization algorithm (QAOA) is a variational method for noisy, intermediate-scale quantum computers to solve combinatorial optimization problems. Quantifying performance bounds with respect to specific problem instances provides insight into when QAOA may be viable for solving real-world applications. Here, we solve every instance of MaxCut on non-isomorphic unweighted graphs with nine or fewer vertices by numerically simulating the pure-state dynamics of QAOA. Testing up to three layers of QAOA depth, we find that distributions of the approximation ratio narrow with increasing depth while the probability of recovering the maximum cut generally broadens. We find QAOA exceeds the Goemans-Williamson approximation ratio bound for most graphs. We also identify consistent patterns within the ensemble of optimized variational circuit parameters that offer highly efficient heuristics for solving MaxCut with QAOA. The resulting data set is presented as a benchmark for establishing empirical bounds on QAOA performance that may be used to test on-going experimental realizations.
Performance of Machine Learning Algorithms with Different K Values in K-fold CrossValidation

The numerical value of k in a k-fold cross-validation training technique of machine learning predictive models is an essential element that impacts the model’s performance. A right choice of k results in better accuracy, while a poorly chosen value for k might affect the model’s performance. In literature, the most commonly used values of k are five (5) or ten (10), as these two values are believed to give test error rate estimates that suffer neither from extremely high bias nor very high variance. However, there is no formal rule. To the best of our knowledge, few experimental studies attempted to investigate the effect of diverse k values in training different machine learning models. This paper empirically analyses the prevalence and effect of distinct k values (3, 5, 7, 10, 15 and 20) on the validation performance of four well-known machine learning algorithms (Gradient Boosting Machine (GBM), Logistic Regression (LR), Decision Tree (DT) and K-Nearest Neighbours (KNN)). It was observed that the value of k and model validation performance differ from one machine-learning algorithm to another for the same classification task. However, our empirical suggest that k = 7 offers a slight increase in validations accuracy and area under the curve measure with lesser computational complexity than k = 10 across most MLA. We discuss in detail the study outcomes and outline some guidelines for beginners in the machine learning field in selecting the best k value and machine learning algorithm for a given task.