Reviewerly/exHarmony huggingface dataset:
Learning Sequential and Structural Information for Source Code Summarization

We propose a model that learns both the sequential and the structural features of code for source code summarization.We adopt the abstract syntax tree (AST) and graph convolution to model the structural information and the Transformer to model the sequential information.We convert code snippets into ASTs and apply graph convolution to obtain structurally-encoded node representations.Then, the sequences of the graphconvolutioned AST nodes are processed by the Transformer layers.Since structurallyneighboring nodes will have similar representations in graph-convolutioned trees, the Transformer layers can effectively capture not only the sequential information but also the structural information such as sentences or blocks of source code.We show that our model outperforms the state-of-the-art for source code summarization by experiments and human evaluations.
Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models

Pre-trained language models have achieved human-level performance on many Machine Reading Comprehension (MRC) tasks, but it remains unclear whether these models truly understand language or answer questions by exploiting statistical biases in datasets.Here, we demonstrate a simple yet effective method to attack MRC models and reveal the statistical biases in these models.We apply the method to the RACE dataset, for which the answer to each MRC question is selected from 4 options.It is found that several pre-trained language models, including BERT, ALBERT, and RoBERTa, show consistent preference to some options, even when these options are irrelevant to the question.When interfered by these irrelevant options, the performance of MRC models can be reduced from human-level performance to the chance-level performance.Human readers, however, are not clearly affected by these irrelevant options.Finally, we propose an augmented training method that can greatly reduce models' statistical biases.
Handwritten Signature Verification Method Based on Improved Combined Features

As a behavior feature, handwritten signatures are widely used in financial and administrative institutions. The appearance of forged signatures will cause great property losses to customers. This paper proposes a handwritten signature verification method based on improved combined features. According to advanced smart pen technology, when writing a signature, offline images and online data of the signature can be obtained in real time. It is the first time to realize the combination of offline and online. We extract the static and dynamic features of the signature and verify them with support vector machine (SVM) and dynamic time warping (DTW) respectively. We use a small number of samples during the training stage, which solves the problem of insufficient number of samples to a certain extent. We get two decision scores while getting the verification result. Finally, we propose a score fusion method based on accuracy (SF-A), which combines offline and online features through score fusion and utilize the complementarity among classifiers effectively. Experimental results show that using different numbers of training samples to conduct experiments on local data sets, the false acceptance rate (FAR) and false reject rate (FRR) obtained are better than the offline or online verification results.
Task Cooperation for Semi-Supervised Few-Shot Learning

Training a model with limited data is an essential task for machine learning and visual recognition. Few-shot learning approaches meta-learn a task-level inductive bias from SEEN class few-shot tasks, and the meta-model is expected to facilitate the few-shot learning with UNSEEN classes. Inspired by the idea that unlabeled data can be utilized to smooth the model space in traditional semi-supervised learning, we propose TAsk COoperation (TACO) which takes advantage of unsupervised tasks to smooth the meta-model space. Specifically, we couple the labeled support set in a few-shot task with easily-collected unlabeled instances, prediction agreement on which encodes the relationship between tasks. The learned smooth meta-model promotes the generalization ability on supervised UNSEEN few-shot tasks. The state-of-the-art few-shot classification results on MiniImageNet and TieredImageNet verify the superiority of TACO to leverage unlabeled data and task relationship in meta-learning.
What are GANs?: Introducing Generative Adversarial Networks to Middle School Students

Applications of Generative Machine Learning techniques such as Generative Adversarial Networks (GANs) are used to generate new instances of images, music, text, and videos. While GANs have now become commonplace on social media, a part of children’s lives, and have considerable ethical implications, existing K-12 AI education curricula do not include generative AI. We present a new module, “What are GANs?”, that teaches middle school students how GANs work and how they can create media using GANs. We developed an online, team-based game to simulate how GANs work. Students also interacted with up to four web tools that apply GANs to generate media. This module was piloted with 72 middle school students in a series of online workshops. We provide insight into student usage, understanding, and attitudes towards this lesson. Finally, we give suggestions for integrating this lesson into AI education curricula.
Discovering Dialog Structure Graph for Coherent Dialog Generation

Learning discrete dialog structure graph from human-human dialogs yields basic insights into the structure of conversation, and also provides background knowledge to facilitate dialog generation.However, this problem is less studied in open-domain dialogue.In this paper, we conduct unsupervised discovery of discrete dialog structure from chitchat corpora, and then leverage it to facilitate coherent dialog generation in downstream systems.To this end, we present an unsupervised model, Discrete Variational Auto-Encoder with Graph Neural Network (DVAE-GNN), to discover discrete hierarchical latent dialog states (at the level of both session and utterance) and their transitions from corpus as a dialog structure graph.Then we leverage it as background knowledge to facilitate dialog management in a RL based dialog system.Experimental results on two benchmark corpora confirm that DVAE-GNN can discover meaningful dialog structure graph, and the use of dialog structure as background knowledge can significantly improve multi-turn coherence.
Decoupling Adversarial Training for Fair NLP

Adversarial debiasing can help to learn fairer models.Previous work has assumed that both main task labels and protected attributes are available in the dataset.However, protected labels are often unavailable, or only available in limited numbers.In this paper, we propose a training strategy which needs only a small volume of protected labels in adversarial training, incorporating an estimation method to transfer private-labelled instances from one dataset to another.We demonstrate the in-and crossdomain effectiveness of our method through a range of experiments.
Semi-Supervised Text Classification with Balanced Deep Representation Distributions

Semi-Supervised Text Classification (SSTC) mainly works under the spirit of self-training.They initialize the deep classifier by training over labeled texts; and then alternatively predict unlabeled texts as their pseudo-labels and train the deep classifier over the mixture of labeled and pseudo-labeled texts.Naturally, their performance is largely affected by the accuracy of pseudo-labels for unlabeled texts.Unfortunately, they often suffer from low accuracy because of the margin bias problem caused by the large difference between representation distributions of labels in SSTC.To alleviate this problem, we apply the angular margin loss, and perform Gaussian linear transformation to achieve balanced label angle variances, i.e., the variance of label angles of texts within the same label.More accuracy of predicted pseudo-labels can be achieved by constraining all label angle variances balanced, where they are estimated over both labeled and pseudo-labeled texts during self-training loops.With this insight, we propose a novel SSTC method, namely Semi-Supervised Text Classification with Balanced Deep representation Distributions (S 2 TC-BDD).To evaluate S 2 TC-BDD, we compare it against the state-of-theart SSTC methods.Empirical results demonstrate the effectiveness of S 2 TC-BDD, especially when the labeled texts are scarce.
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units

In this paper we present the first model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision.Instead, we connect the image captioning module and the speech synthesis module with a set of discrete, sub-word speech units that are discovered with a self-supervised visual grounding task.We conduct experiments on the Flickr8k spoken caption dataset in addition to a novel corpus of spoken audio captions collected for the popular MSCOCO dataset, demonstrating that our generated captions also capture diverse visual semantics of the images they describe.We investigate several different intermediate speech representations, and empirically find that the representation must satisfy several important properties to serve as drop-in replacements for text.
Accelerating BERT Inference for Sequence Labeling via Early-Exit

Both performance and efficiency are crucial factors for sequence labeling tasks in many real-world scenarios.Although the pre-trained models (PTMs) have significantly improved the performance of various sequence labeling tasks, their computational cost is expensive.To alleviate this problem, we extend the recent successful early-exit mechanism to accelerate the inference of PTMs for sequence labeling tasks.However, existing early-exit mechanisms are specifically designed for sequencelevel tasks, rather than sequence labeling.In this paper, we first propose SENTEE: a simple extension of SENTence-level Early-Exit for sequence labeling tasks.To further reduce computational cost, we also propose TOKEE: a TOKen-level Early-Exit mechanism that allows partial tokens to exit early at different layers.Considering the local dependency inherent in sequence labeling, we employed a window-based criterion to decide for a token whether or not to exit.The token-level earlyexit brings the gap between training and inference, so we introduce an extra self-sampling fine-tuning stage to alleviate it.The extensive experiments on three popular sequence labeling tasks show that our approach can save up to 66%∼75% inference cost with minimal performance degradation.Compared with competitive compressed models such as DistilBERT, our approach can achieve better performance under the same speed-up ratios of 2×, 3×, and 4×. 1
Improving Unsupervised Extractive Summarization with Facet-Aware Modeling

Unsupervised extractive summarization aims to extract salient sentences from documents without labeled corpus.Existing methods are mostly graph-based by computing sentence centrality.These methods usually tend to select sentences within the same facet, however, which often leads to the facet bias problem especially when the document has multiple facets (i.e.long-document and multidocuments).To address this problem, we proposed a novel facet-aware centrality-based ranking model.We let the model pay more attention to different facets by introducing a sentence-document weight.The weight is added to the sentence centrality score.We evaluate our method on a wide range of summarization tasks that include 8 representative benchmark datasets.Experimental results show that our method consistently outperforms strong baselines especially in longand multi-document scenarios and even performs comparably to some supervised models.Extensive analyses confirm that the performance gains come from alleviating the facet bias problem.
On the Adequacy of Untuned Warmup for Adaptive Optimization

Adaptive optimization algorithms such as Adam (Kingma and Ba, 2014) are widely used in deep learning. The stability of such algorithms is often improved with a warmup schedule for the learning rate. Motivated by the difficulty of choosing and tuning warmup schedules, recent work proposes automatic variance rectification of Adam's adaptive learning rate, claiming that this rectified approach ("RAdam") surpasses the vanilla Adam algorithm and reduces the need for expensive tuning of Adam with warmup. In this work, we refute this analysis and provide an alternative explanation for the necessity of warmup based on the magnitude of the update term, which is of greater relevance to training stability. We then provide some "rule-of-thumb" warmup schedules, and we demonstrate that simple untuned warmup of Adam performs more-or-less identically to RAdam in typical practical settings. We conclude by suggesting that practitioners stick to linear warmup with Adam, with a sensible default being linear warmup over 2 / (1 - β₂) training iterations.
Lexicon Learning for Few Shot Sequence Modeling

Sequence-to-sequence transduction is the core problem in language processing applications as diverse as semantic parsing, machine translation, and instruction following.The neural network models that provide the dominant solution to these problems are brittle, especially in low-resource settings: they fail to generalize correctly or systematically from small datasets.Past work has shown that many failures of systematic generalization arise from neural models' inability to disentangle lexical phenomena from syntactic ones.To address this, we augment neural decoders with a lexical translation mechanism that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules.We describe how to initialize this mechanism using a variety of lexicon learning algorithms, and show that it improves systematic generalization on a diverse set of sequence modeling tasks drawn from cognitive science, formal semantics, and machine translation.
Movie Summarization via Sparse Graph Construction

We summarize full-length movies by creating shorter videos containing their most informative scenes. We explore the hypothesis that a summary can be created by assembling scenes which are turning points (TPs), i.e., key events in a movie that describe its storyline. We propose a model that identifies TP scenes by building a sparse movie graph that represents relations between scenes and is constructed using multimodal information. According to human judges, the summaries created by our approach are more informative and complete, and receive higher ratings, than the outputs of sequence-based models and general-purpose summarization algorithms. The induced graphs are interpretable, displaying different topology for different movie genres.
Agent Incentives: A Causal Perspective

We present a framework for analysing agent incentives using causal influence diagrams. We establish that a well-known criterion for value of information is complete. We propose a new graphical criterion for value of control, establishing its soundness and completeness. We also introduce two new concepts for incentive analysis: response incentives indicate which changes in the environment affect an optimal decision, while instrumental control incentives establish whether an agent can influence its utility via a variable X. For both new concepts, we provide sound and complete graphical criteria. We show by example how these results can help with evaluating the safety and fairness of an AI system
Entity-Aware Abstractive Multi-Document Summarization

Entities and their mentions convey significant semantic information in documents.In multidocument summarization, the same entity may appear across different documents.Capturing such cross-document entity information can be beneficial -intuitively, it allows the system to aggregate diverse useful information around the same entity for better summarization.In this paper, we present EMSum, an entityaware model for abstractive multi-document summarization.Our model augments the classical Transformer-based encoder-decoder framework with a heterogeneous graph consisting of text units and entities as nodes, which allows rich cross-document information to be captured.In the decoding process, we design a novel two-level attention mechanism, allowing the model to deal with saliency and redundancy issues explicitly.Our model can also be used together with pre-trained language models, arriving at improved performance.We conduct comprehensive experiments on the standard datasets and the results show the effectiveness of our approach.
Grounding ‘Grounding’ in NLP

The NLP community has seen substantial recent interest in grounding to facilitate interaction between language technologies and the world.However, as a community, we use the term broadly to reference any linking of text to data or non-textual modality.In contrast, Cognitive Science more formally defines "grounding" as the process of establishing what mutual information is required for successful communication between two interlocutorsa definition which might implicitly capture the NLP usage but differs in intent and scope.We investigate the gap between these definitions and seek answers to the following questions: (1) What aspects of grounding are missing from NLP tasks?Here we present the dimensions of coordination, purviews and constraints.(2) How is the term "grounding" used in the current research?We study the trends in datasets, domains, and tasks introduced in recent NLP conferences.And finally, (3) How to advance our current definition to bridge the gap with Cognitive Science?We present ways to both create new tasks or repurpose existing ones to make advancements towards achieving a more complete sense of grounding.github.com/khyathiraghavi/Grounding-Grounding
Multi-scale graph attention subspace clustering network

Deep subspace clustering (DSC) network which uses deep auto-encoders (DAE) mapping raw data into a latent space for clustering has achieved significant performance. However, when it comes to graph-based clustering, it fails to encode the node attribute and graph structure simultaneously, which degrades the clustering performance. Meanwhile, existing graph convolutional network (GCN) based clustering methods usually do not have a clustering-oriented objection function since the process of learning representation and clustering are separated. In addition, both aforementioned methods can neither catch the multi-scale information nor use the current clustering labels effectively, which lead to the suboptimal performance. To this end, we proposed a novel end-to-end framework called Multi-Scale Graph Attention Subspace Clustering Network (MSGA). By employing a novel GCN-based feature extraction module, it can effectively capture the node representation on graph-based datasets. Moreover, a multi-scale self-expression module is designed for obtaining a more discriminative coefficient representation from each layer of the encoder and a self-supervised module is introduced for supervising the learning of node representation. Specifically, we train and optimize them in a unified framework so that different modules can benefit from each other. Extensive experiments demonstrate the superiority of our method compared with several state-of-the-art methods.
Self-Supervised Attention-Aware Reinforcement Learning

Visual saliency has emerged as a major visualization tool for interpreting deep reinforcement learning (RL) agents. However, much of the existing research uses it as an analyzing tool rather than an inductive bias for policy learning. In this work, we use visual attention as an inductive bias for RL agents. We propose a novel self-supervised attention learning approach which can 1. learn to select regions of interest without explicit annotations, and 2. act as a plug for existing deep RL methods to improve the learning performance. We empirically show that the self-supervised attention-aware deep RL methods outperform the baselines in the context of both the rate of convergence and performance. Furthermore, the proposed self-supervised attention is not tied with specific policies, nor restricted to a specific scene. We posit that the proposed approach is a general self-supervised attention module for multi-task learning and transfer learning, and empirically validate the generalization ability of the proposed method. Finally, we show that our method learns meaningful object keypoints highlighting improvements both qualitatively and quantitatively.
Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level

Larger language models have higher accuracy on average, but are they better on every single instance (datapoint)?Some work suggests larger models have higher out-ofdistribution robustness, while other work suggests they have lower accuracy on rare subgroups.To understand these differences, we investigate these models at the level of individual instances.However, one major challenge is that individual predictions are highly sensitive to noise in the randomness in training.We develop statistically rigorous methods to address this, and after accounting for pretraining and finetuning noise, we find that our BERT-LARGE is worse than BERT-MINI on at least 1-4% of instances across MNLI, SST-2, and QQP, compared to the overall accuracy improvement of 2-10%.We also find that finetuning noise increases with model size, and that instance-level accuracy has momentum: improvement from BERT-MINI to BERT-MEDIUM correlates with improvement from BERT-MEDIUM to BERT-LARGE .Our findings suggest that instance-level predictions provide a rich source of information; we therefore recommend that researchers supplement model weights with model predictions.
Revisiting the Negative Data of Distantly Supervised Relation Extraction

Distantly supervision automatically generates plenty of training samples for relation extraction.However, it also incurs two major problems: noisy labels and imbalanced training data.Previous works focus more on reducing wrongly labeled relations (false positives) while few explore the missing relations that are caused by incompleteness of knowledge base (false negatives).Furthermore, the quantity of negative labels overwhelmingly surpasses the positive ones in previous problem formulations.In this paper, we first provide a thorough analysis of the above challenges caused by negative data.Next, we formulate the problem of relation extraction into as a positive unlabeled learning task to alleviate false negative problem.Thirdly, we propose a pipeline approach, dubbed RERE, that first performs sentence classification with relational labels and then extracts the subjects/objects.Experimental results show that the proposed method consistently outperforms existing approaches and remains excellent performance even learned with a large quantity of false positive samples.Source code is available online 1 .
Factorising Meaning and Form for Intent-Preserving Paraphrasing

We propose a method for generating paraphrases of English questions that retain the original intent but use a different surface form.Our model combines a careful choice of training objective with a principled information bottleneck, to induce a latent encoding space that disentangles meaning and form.We train an encoder-decoder model to reconstruct a question from a paraphrase with the same meaning and an exemplar with the same surface form, leading to separated encoding spaces.We use a Vector-Quantized Variational Autoencoder to represent the surface form as a set of discrete latent variables, allowing us to use a classifier to select a different surface form at test time.Crucially, our method does not require access to an external source of target exemplars.Extensive experiments and a human evaluation show that we are able to generate paraphrases with a better tradeoff between semantic preservation and syntactic novelty compared to previous methods. Paralex