Reviewerly/exHarmony huggingface dataset:
Deep direct likelihood knockoffs.

Predictive modeling often uses black box machine learning methods, such as deep neural networks, to achieve state-of-the-art performance. In scientific domains, the scientist often wishes to discover which features are actually important for making the predictions. These discoveries may lead to costly follow-up experiments and as such it is important that the error rate on discoveries is not too high. Model-X knockoffs [2] enable important features to be discovered with control of the false discovery rate (fdr). However, knockoffs require rich generative models capable of accurately modeling the knockoff features while ensuring they obey the so-called "swap" property. We develop Deep Direct Likelihood Knockoffs (ddlk), which directly minimizes the KL divergence implied by the knockoff swap property. ddlk consists of two stages: it first maximizes the explicit likelihood of the features, then minimizes the KL divergence between the joint distribution of features and knockoffs and any swap between them. To ensure that the generated knockoffs are valid under any possible swap, ddlk uses the Gumbel-Softmax trick to optimize the knockoff generator under the worst-case swap. We find ddlk has higher power than baselines while controlling the false discovery rate on a variety of synthetic and real benchmarks including a task involving a large dataset from one of the epicenters of COVID-19.
Image GANs meet Differentiable Rendering for Inverse Graphics and  Interpretable 3D Neural Rendering

Differentiable rendering has paved the way to training neural networks to perform "inverse graphics" tasks such as predicting 3D geometry from monocular photographs. To train high performing models, most of the current approaches rely on multi-view imagery which are not readily available in practice. Recent Generative Adversarial Networks (GANs) that synthesize images, in contrast, seem to acquire 3D knowledge implicitly during training: object viewpoints can be manipulated by simply manipulating the latent codes. However, these latent codes often lack further physical interpretation and thus GANs cannot easily be inverted to perform explicit 3D reasoning. In this paper, we aim to extract and disentangle 3D knowledge learned by generative models by utilizing differentiable renderers. Key to our approach is to exploit GANs as a multi-view data generator to train an inverse graphics network using an off-the-shelf differentiable renderer, and the trained inverse graphics network as a teacher to disentangle the GAN's latent code into interpretable 3D properties. The entire architecture is trained iteratively using cycle consistency losses. We show that our approach significantly outperforms state-of-the-art inverse graphics networks trained on existing datasets, both quantitatively and via user studies. We further showcase the disentangled GAN as a controllable 3D "neural renderer", complementing traditional graphics renderers.
Revisiting Temporal Modeling for Video Super-resolution

Video super-resolution plays an important role in surveillance video analysis and ultra-high-definition video display, which has drawn much attention in both the research and industrial communities. Although many deep learning-based VSR methods have been proposed, it is hard to directly compare these methods since the different loss functions and training datasets have a significant impact on the super-resolution results. In this work, we carefully study and compare three temporal modeling methods (2D CNN with early fusion, 3D CNN with slow fusion and Recurrent Neural Network) for video super-resolution. We also propose a novel Recurrent Residual Network (RRN) for efficient video super-resolution, where residual learning is utilized to stabilize the training of RNN and meanwhile to boost the super-resolution performance. Extensive experiments show that the proposed RRN is highly computational efficiency and produces temporal consistent VSR results with finer details than other temporal modeling methods. Besides, the proposed method achieves state-of-the-art results on several widely used benchmarks.
A novel locally-constrained GAN-based ensemble to synthesize arterial spin labeling images

Arterial spin labeling (ASL) images that are capable to quantitatively measure the cerebral blood flow receive increasing research attention in recent dementia diagnosis studies. However, this important imaging modality is unfortunately not commonly seen in many well-established image-based dementia datasets. Hence, synthesizing ASL images to supplement the important modality in these datasets for further improving the accuracy of dementia diseases diagnosis is quite important and valuable. In this study, a novel locally-constrained generative adversarial networks (GAN)-based ensemble is introduced to fulfill the ASL image synthesis task for improving the dementia diseases diagnosis performance. Technically, new attention-based feature pyramid-GAN models are designed as local models of the novel ensemble. Also, multi-Gaussian-distributed noise is generated from a new flow-based generative model and utilized in medical image synthesis, for the first time. Experiments have been conducted to reveal the effectiveness of the novel GAN ensemble. Comparisons between the novel GAN ensemble and many other state-of-the-art methods in medical image synthesis have been carried out. Statistical analyses have suggested that, accuracies of dementia diseases diagnosis can be significantly improved with the help of the novel GAN ensemble, which brings about 41.62% performance improvement based on a 355-demented-patient dataset and approximately 25% performance improvement from the well-known ADNI-1 dataset.
Cross-Lingual QA as a Stepping Stone for Monolingual Open QA in Icelandic

It can be challenging to build effective open question answering (open QA) systems for languages other than English, mainly due to a lack of labeled data for training.We present a data efficient method to bootstrap such a system for languages other than English.Our approach requires only limited QA resources in the given language, along with machine-translated data, and at least a bilingual language model.To evaluate our approach, we build such a system for the Icelandic language and evaluate performance over trivia style datasets.The corpora used for training are English in origin but machine translated into Icelandic.We train a bilingual Icelandic/English language model to embed English context and Icelandic questions following methodology introduced with DensePhrases (Lee et al., 2021).The resulting system is an open domain cross-lingual QA system between Icelandic and English.Finally, the system is adapted for Icelandic only open QA, demonstrating how it is possible to efficiently create an open QA system with limited access to curated datasets in the language of interest.
Improving Classification of Infrequent Cognitive Distortions: Domain-Specific Model vs. Data Augmentation

Cognitive distortions are counterproductive patterns of thinking that are one of the targets of cognitive behavioral therapy (CBT).These can be challenging for clinicians to detect, especially those without extensive CBT training or supervision.Text classification methods can approximate expert clinician judgment in the detection of frequently occurring cognitive distortions in text-based therapy messages.However, performance with infrequent distortions is relatively poor.In this study, we address this sparsity problem with two approaches: Data Augmentation and Domain-Specific Model.The first approach includes Easy Data Augmentation, back translation, and mixup techniques.The second approach utilizes a domain-specific pretrained language model, MentalBERT.To examine the viability of different data augmentation methods, we utilized a real-world dataset of texts between therapists and clients diagnosed with serious mental illness that was annotated for distorted thinking.We found that with optimized parameter settings, mixup was helpful for rare classes.Performance improvements with an augmented model, MentalBERT, exceed those obtained with data augmentation.
Paragraph-based Transformer Pre-training for Multi-Sentence Inference

Inference tasks such as answer sentence selection (AS2) or fact verification are typically solved by fine-tuning transformer-based models as individual sentence-pair classifiers.Recent studies show that these tasks benefit from modeling dependencies across multiple candidate sentences jointly.In this paper, we first show that popular pre-trained transformers perform poorly when used for fine-tuning on multi-candidate inference tasks.We then propose a new pre-training objective that models the paragraph-level semantics across multiple input sentences.Our evaluation on three AS2 and one fact verification datasets demonstrates the superiority of our pre-training technique over the traditional ones for transformers used as joint models for multi-candidate inference tasks, as well as when used as cross-encoders for sentence-pair formulations of these tasks.
Xu at SemEval-2022 Task 4: Pre-BERT Neural Network Methods vs Post-BERT RoBERTa Approach for Patronizing and Condescending Language Detection

This paper describes my participation in the SemEval-2022 Task 4: Patronizing and Condescending Language Detection.I participate in both subtasks: Patronizing and Condescending Language (PCL) Identification and Patronizing and Condescending Language Categorization, with the main focus put on subtask 1.The experiments compare pre-BERT neural network (NN) based systems against post-BERT pretrained language model RoBERTa.This research finds that NN-based systems in the experiments perform worse on the task compared to the pretrained language models.The topperforming RoBERTa system is ranked 26 out of 78 teams (F1-score: 54.64) in subtask 1, and 23 out of 49 teams (F1-score: 30.03) in subtask 2.
DACSA: A large-scale Dataset for Automatic summarization of Catalan and Spanish newspaper Articles

The application of supervised methods to automatic summarization requires the availability of adequate corpora consisting of a set of document-summary pairs.As in most Natural Language Processing tasks, the great majority of available datasets for summarization are in English, making it difficult to develop automatic summarization models for other languages.Although Spanish is gradually forming part of some recent summarization corpora, it is not the same for minority languages such as Catalan.In this work, we describe the construction of a corpus of Catalan and Spanish newspapers, the Dataset for Automatic summarization of Catalan and Spanish newspaper Articles (DACSA) corpus.It is a high-quality large-scale corpus that can be used to train summarization models for Catalan and Spanish.We have carried out an analysis of the corpus, both in terms of the style of the summaries and the difficulty of the summarization task.In particular, we have used a set of well-known metrics in the summarization field in order to characterize the corpus.Additionally, we have evaluated the performance of some extractive and abstractive summarization systems on the DACSA corpus for benchmarking purposes.
Semantic-Preserving Abstractive Text Summarization with Siamese Generative Adversarial Net

We propose a novel siamese generative adver sarial net for abstractive text summarization (SSPGAN), which can preserve the main se mantics of the source text.Different from pre vious generative adversarial net based methods, SSPGAN is equipped with a siamese semanticpreserving discriminator, which can not only be trained to discriminate the machine-generated summaries from the human-summarized ones, but also ensure the semantic consistency be tween the source text and target summary.As a consequence of the min-max game between the generator and the siamese semantic-preserving discriminator, the generator can generate a sum mary that conveys the key content of the source text more accurately.Extensive experiments on several text summarization benchmarks in dif ferent languages demonstrate the effectiveness of the proposed method.* Corresponding author et al., 2015) is proposed to strengthen the sequenceto-sequence models, which enables the model to focus on the relevant parts of the source-side sequence.Based on the encoder-decoder framework, many variants of model structures, such as convo lutional neural network (CNN) and recurrent neu ral network (RNN) are proposed (Bahdanau et al., 2014;Gehring et al., 2017).With the emergence of Transformer (Vaswani et al., 2017), which is based entirely on the attention mechanism, state-of-the art performance is achieved on many sequence-to sequence tasks.Nevertheless, for the task of abstractive text summarization, one of the dominant challenges is to maintain saliency, which requires the generated summary to convey the important information accurately.As shown in Figure 1, the key content of the source text "Chengdu become the 'Silicon Valley' in the west of our country" is
A New Concept of Knowledge based Question Answering (KBQA) System for Multi-hop Reasoning

Knowledge based question answering (KBQA) is a complex task for natural language understanding.Many KBQA approaches have been proposed in recent years, and most of them are trained based on labeled reasoning path.This hinders the system's performance as many correct reasoning paths are not labeled as ground truth, and thus they cannot be learned.In this paper, we introduce a new concept of KBQA system which can leverage multiple reasoning paths' information and only requires labeled answer as supervision.We name it as Mutliple Reasoning Paths KBQA System (MRP-QA).We conduct experiments on several benchmark datasets containing both singlehop simple questions as well as muti-hop complex questions, including WebQues-tionSP (WQSP), ComplexWebQuestion-1.1 (CWQ), and PathQuestion-Large (PQL), and demonstrate strong performance.
DisTA: Generic Dynamic Taint Tracking for Java-Based Distributed Systems

Dynamic taint tracking is a powerful information flow analysis approach, which can be applied in many analysis scenarios, e.g., debugging, testing, and security vulnerability detection. Most dynamic taint tracking approaches are designed for standalone systems, and cannot support inter-node taint tracking in distributed systems. Few inter-node taint tracking approaches are designed for specific distributed systems, e.g., Apache Spark, and require specific modifications to different distributed systems.In this paper, we present DisTA, a generic dynamic taint tracking tool for Java-based distributed systems. By instrumenting common network communication modules in Java, DisTA can perform inter-node taint tracking for different distributed systems with little manual efforts. We evaluate DisTA on five large-scale real-world distributed systems, e.g., ZooKeeper and Yarn, and require only 10 LOC launch script modification on average. The experimental results show that DisTA can accurately track all inter-node taints with a relatively low overhead.
MINION: a Large-Scale and Diverse Dataset for Multilingual Event Detection

Event Detection (ED) is the task of identifying and classifying trigger words of event mentions in text.Despite considerable research efforts in recent years for English text, the task of ED in other languages has been significantly less explored.Switching to non-English languages, important research questions for ED include how well existing ED models perform on different languages, how challenging ED is in other languages, and how well ED knowledge and annotation can be transferred across languages.To answer those questions, it is crucial to obtain multilingual ED datasets that provide consistent event annotation for multiple languages.There exist some multilingual ED datasets; however, they tend to cover a handful of languages and mainly focus on popular ones.Many languages are not covered in existing multilingual ED datasets.In addition, the current datasets are often small and not accessible to the public.To overcome those shortcomings, we introduce a new large-scale multilingual dataset for ED (called MINION) that consistently annotates events for 8 different languages; 5 of them have not been supported by existing multilingual datasets.We also perform extensive experiments and analysis to demonstrate the challenges and transferability of ED across languages in MINION that in all call for more research effort in this area.We will release the dataset to promote future research on multilingual ED.
Yes, No or IDK: The Challenge of Unanswerable Yes/No Questions

The Yes/No QA task (Clark et al., 2019) consists of "Yes" or "No" questions about a given context.However, in realistic scenarios, the information provided in the context is not always sufficient in order to answer the question.For example, given the context "She married a lawyer from New-York.",we don't know whether the answer to the question "Did she marry in New York?" is "Yes" or "No".In this paper, we extend the Yes/No QA task, adding questions with an IDK answer, and show its considerable difficulty compared to the original 2-label task.For this purpose, we (i) enrich the BoolQ dataset (Clark et al., 2019) to include unanswerable questions and (ii) create out-ofdomain test sets for the Yes/No/IDK QA task.We study the contribution of training on other Natural Language Understanding tasks.We focus in particular on Extractive QA (Rajpurkar et al., 2018) and Recognizing Textual Entailments (RTE, Dagan et al., 2013), analyzing the differences between 2 and 3 labels using the new data. 1
MMVAE at SemEval-2022 Task 5: A Multi-modal Multi-task VAE on Misogynous Meme Detection

Memes have become quite common in day-today communications on social media platforms.They often appear to be amusing, evoking and attractive to audiences.However, some memes containing malicious content can be harmful to targeted groups.In this paper, we study misogynous meme detection, a shared task in SemEval 2022 -Multimedia Automatic Misogyny Identification (MAMI).The challenge of misogynous meme detection is to co-represent multi-modal features.To tackle with this challenge, we propose a Multi-modal Multi-task Variational Au-toEncoder (MMVAE) to learn an effective corepresentation of visual and textual features in the latent space.Our goal is to automatically determine if a meme contains misogynous information and then identify its fine-grained category.Our model achieves F 1 scores of 0.723 on the MAMI sub-task A and 0.634 on sub-task B. We carry out comprehensive experiments on our model's architecture and show that our approach significantly outperforms several strong uni-modal and multi-modal approaches.Our code is released on github 1 .
Multi-Relational Graph Transformer for Automatic Short Answer Grading

The recent transition to the online educational domain has increased the need for Automatic Short Answer Grading (ASAG).ASAG automatically evaluates a student's response against a (given) correct response and thus has been a prevalent semantic matching task.Most existing methods utilize sequential context to compare two sentences and ignore the structural context of the sentence; therefore, these methods may not result in the desired performance.In this paper, we overcome this problem by proposing a Multi-Relational Graph Transformer, MitiGaTe, to prepare token representations considering the structural context.Abstract Meaning Representation (AMR) graph is created by parsing the text response and then segregated into multiple subgraphs, each corresponding to a particular relationship in AMR.A Graph Transformer is used to prepare relation-specific token embeddings within each subgraph, then aggregated to obtain a subgraph representation.Finally, we compare the correct answer and the student response subgraph representations to yield a final score.Experimental results on Mohler's dataset show that our system outperforms the existing stateof-the-art methods.We have released our implementation 1 , as we believe that our model can be useful for many future applications.
IIT DHANBAD CODECHAMPS at SemEval-2022 Task 5: MAMI - Multimedia Automatic Misogyny Identification

With the growth of the internet, the use of social media based on images has drastically increased like Twitter, Instagram, etc.In these social media, women have a very high contribution as of 75% women use social media multiple times compared to men which is only 65% of men uses social media multiple times a day.However, with this much contribution, it also increases systematic inequality and discrimination offline is replicated in online spaces in the form of MEMEs.A meme is essentially an image characterized by pictorial content with an overlaying text a posteriori introduced by humans, with the main goal of being funny and/or ironic.Although most of them are created with the intent of making funny jokes, in a short time people started to use them as a form of hate and prejudice against women, landing to sexist and aggressive messages in online environments that subsequently amplify the sexual stereotyping and gender inequality of the offline world.This leads to the need for automatic detection of Misogyny MEMEs.Specifically, I described the model submitted for the shared task on Multimedia Automatic Misogyny Identification (MAMI (Fersini et al., 2022)) and my team name is IIT DHANBAD CODECHAMPS.
Enhancing Self-Attention with Knowledge-Assisted Attention Maps

Large-scale pre-trained language models have attracted extensive attentions in the research community and shown promising results on various tasks of natural language processing.However, the attention maps, which record the attention scores between tokens in self-attention mechanism, are sometimes ineffective as they are learned implicitly without the guidance of explicit semantic knowledge.Thus, we aim to infuse explicit external knowledge into pretrained language models to further boost their performance.Existing works of knowledge infusion largely depend on multi-task learning frameworks, which are inefficient and require large-scale re-training when new knowledge is considered.In this paper, we propose a novel and generic solution, KAM-BERT, which directly incorporates knowledge-generated attention maps into the self-attention mechanism.It requires only a few extra parameters and supports efficient fine-tuning once new knowledge is added.KAM-BERT achieves consistent improvements on various academic datasets for natural language understanding.It also outperforms other state-of-the-art methods which conduct knowledge infusion into transformerbased architectures.Moreover, we apply our model to an industry-scale ad relevance application and show its advantages in the real-world scenario.
Exploring the Effect of Dialect Mismatched Language Models in Telugu Automatic Speech Recognition

Previous research has found that Acoustic Models (AM) of an Automatic Speech Recognition (ASR) system are susceptible to dialect variations within a language, thereby adversely affecting the ASR.To counter this, researchers have proposed to build a dialectspecific AM while keeping the Language Model (LM) constant for all the dialects.This study explores the effect of dialect mismatched LM by considering three different Telugu regional dialects: Telangana, Coastal Andhra, and Rayalaseema.We show that dialect variations that surface in the form of a different lexicon, grammar, and occasionally semantics can significantly degrade the performance of the LM under mismatched conditions.Therefore, this degradation has an adverse effect on the ASR even when dialect-specific AM is used.We show a degradation of up to 13.13 perplexity points when LM is used under mismatched conditions.Furthermore, we show a degradation of over 9% and over 15% in Character Error Rate (CER) and Word Error Rate (WER), respectively, in the ASR systems when using mismatched LMs over matched LMs.