Reviewerly/exHarmony huggingface dataset:
Lite Transformer with Long-Short Range Attention

Transformer has become ubiquitous in natural language processing (e.g., machine translation, question answering); however, it requires enormous amount of computations to achieve high performance, which makes it not suitable for mobile applications that are tightly constrained by the hardware resources and battery. In this paper, we present an efficient mobile NLP architecture, Lite Transformer to facilitate deploying mobile NLP applications on edge devices. The key primitive is the Long-Short Range Attention (LSRA), where one group of heads specializes in the local context modeling (by convolution) while another group specializes in the long-distance relationship modeling (by attention). Such specialization brings consistent improvement over the vanilla transformer on three well-established language tasks: machine translation, abstractive summarization, and language modeling. Under constrained resources (500M/100M MACs), Lite Transformer outperforms transformer on WMT'14 English-French by 1.2/1.7 BLEU, respectively. Lite Transformer reduces the computation of transformer base model by 2.5x with 0.3 BLEU score degradation. Combining with pruning and quantization, we further compressed the model size of Lite Transformer by 18.2x. For language modeling, Lite Transformer achieves 1.8 lower perplexity than the transformer at around 500M MACs. Notably, Lite Transformer outperforms the AutoML-based Evolved Transformer by 0.5 higher BLEU for the mobile NLP setting without the costly architecture search that requires more than 250 GPU years. Code has been made available at https://github.com/mit-han-lab/lite-transformer.
Security Analysis of the Image Encryption Algorithm Based on a Two-Dimensional Infinite Collapse Map

This paper analyzes the security of the image encryption algorithm based on a two-dimensional (2D) infinite collapse map. The encryption algorithm adopts a permutation–diffusion structure and can perform two or more rounds to achieve a higher level of security. By cryptanalysis, it is found that the original diffusion process can be split into a permutation–diffusion structure, which comes after the original permutation, so these two permutations can be merged into one. Then, some theorems about round-down operation are summarized, and the encryption and decryption equations in the diffusion process are deduced and simplified accordingly. Since the chaotic sequences used in encryption algorithm are independent of the plaintext and ciphertext, there are equivalent keys. The original encryption algorithm with single-round, two-round, and multi-round of permutation–diffusion processes is cracked, and the data complexity of the cryptanalysis attacks is analyzed. Numerical simulation is carried out by MATLAB, and the experimental results and theoretical analysis show the effectiveness of the cryptanalysis attacks. Finally, some suggestions for improvement are given to overcome the shortcomings of the original encryption algorithm.
Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models

With a growing number of BERTology works analyzing different components of pre-trained language models, we extend this line of research through an in-depth analysis of discourse information in pre-trained and finetuned language models.We move beyond prior work along three dimensions: First, we describe a novel approach to infer discourse structures from arbitrarily long documents.Second, we propose a new type of analysis to explore where and how accurately intrinsic discourse is captured in the BERT and BART models.Finally, we assess how similar the generated structures are to a variety of baselines as well as their distributions within and between models.
Crossroads, Buildings and Neighborhoods: A Dataset for Fine-grained Location Recognition

General domain Named Entity Recognition (NER) datasets like CoNLL-2003 mostly annotate coarse-grained location entities such as a country or a city.But many applications require identifying fne-grained locations from texts and mapping them precisely to geographic sites, e.g., a crossroad, an apartment building, or a grocery store.In this paper, we introduce a new dataset HarveyNER with fne-grained locations annotated in tweets.This dataset presents unique challenges and characterizes many complex and long location mentions in informal descriptions.We built strong baseline models using Curriculum Learning and experimented with different heuristic curricula to better recognize diffcult location mentions.Experimental results show that the simple curricula can improve the system's performance on hard cases and its overall performance, and outperform several other baseline systems.The dataset and the baseline models can be found at https: //github.com/brickee/HarveyNER.
Deep Attention Aware Feature Learning for Person Re-Identification

Visual attention has proven to be effective in improving the performance of person re-identification. Most existing methods apply visual attention heuristically by learning an additional attention map to re-weight the feature maps for person re-identification. However, this kind of methods inevitably increase the model complexity and inference time. In this paper, we propose to incorporate the attention learning as additional objectives in a person ReID network without changing the original structure, thus maintain the same inference time and model size. Two kinds of attentions have been considered to make the learned feature maps being aware of the person and related body parts respectively. Globally, a holistic attention branch (HAB) makes the feature maps obtained by backbone focus on persons so as to alleviate the influence of background. Locally, a partial attention branch (PAB) makes the extracted features be decoupled into several groups and be separately responsible for different body parts (i.e., keypoints), thus increasing the robustness to pose variation and partial occlusion. These two kinds of attentions are universal and can be incorporated into existing ReID networks. We have tested its performance on two typical networks (TriNet and Bag of Tricks) and observed significant performance improvement on five widely used datasets.
NewsEdits: A News Article Revision Dataset and a Novel Document-Level Reasoning Challenge

News article revision histories provide clues to narrative and factual evolution in news articles.To facilitate analysis of this evolution, we present the first publicly available dataset of news revision histories, NewsEdits.Our dataset is large-scale and multilingual; it contains 1.2 million articles with 4.6 million versions from over 22 English-and French-language newspaper sources based in three countries, spanning 15 years of coverage (2006)(2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016)(2017)(2018)(2019)(2020)(2021). 1 We define article-level edit actions: Addition, Deletion, Edit and Refactor, and develop a highaccuracy extraction algorithm to identify these actions.To underscore the factual nature of many edit actions, we conduct analyses showing that added and deleted sentences are more likely to contain updating events, main content and quotes than unchanged sentences.Finally, to explore whether edit actions are predictable, we introduce three novel tasks aimed at predicting actions performed during version updates.We show that these tasks are possible for expert humans but are challenging for large NLP models.We hope this can spur research in narrative framing and help provide predictive tools for journalists chasing breaking news.BERT-Based Subsequence Matching BLEU-Based Method F1-Score Method F1-Score Method F1-Score Hungarian TB-mini 88.5 ngram-1 86.0 BLEU-1 86.7 TB-medium 88.7 ngram-2 88.7 BLEU-2 89.2 RB-base 88.6 ngram-3 88.5 BLEU-3 88.8 Max TB-mini 89.0 ngram-4 88.2 BLEU-1,2 88.8 TB-medium 89.5 BLEU-1,2,3 89.1 RB-base 89.4
Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking

Despite the recent advances in abstractive summarization systems, it is still difficult to determine whether a generated summary is factual consistent with the source text.To this end, the latest approach is to train a factual consistency classifier on factually consistent and inconsistent summaries.Luckily, the former is readily available as reference summaries in existing summarization datasets.However, generating the latter remains a challenge, as they need to be factually inconsistent, yet closely relevant to the source text to be effective.In this paper, we propose to generate factually inconsistent summaries using source texts and reference summaries with key information masked.Experiments on seven benchmark datasets demonstrate that factual consistency classifiers trained on summaries generated using our method generally outperform existing models and show a competitive correlation with human judgments.We also analyze the characteristics of the summaries generated using our method.We will release the pre-trained model and the code at https:// github.com/hwanheelee1993/MFMA.
Aligning to Social Norms and Values in Interactive Narratives

We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based gamesenvironments wherein an agent perceives and interacts with a world through natural language.Such interactive agents are often trained via reinforcement learning to optimize task performance, even when such rewards may lead to agent behaviors that violate societal normscausing harm either to the agent itself or other entities in the environment.Social value alignment refers to creating agents whose behaviors conform to expected moral and social norms for a given context and group of people-in our case, it means agents that behave in a manner that is less harmful and more beneficial for themselves and others.We build on the Jiminy Cricket benchmark (Hendrycks et al., 2021b), a set of 25 annotated interactive narratives containing thousands of morally salient scenarios covering everything from theft and bodily harm to altruism.We introduce the GALAD (Game-value Alignment through Action Distillation) agent that uses the social commonsense knowledge present in specially trained language models to contextually restrict its action space to only those actions that are aligned with socially beneficial values.An experimental study shows that the GALAD agent makes decisions efficiently enough to improve state-of-the-art task performance by 4% while reducing the frequency of socially harmful behaviors by 25% compared to strong contemporary value alignment approaches.
University of Hildesheim at SemEval-2022 task 5: Combining Deep Text and Image Models for Multimedia Misogyny Detection

This paper describes the participation of the University of Hildesheim at the SemEval task 5.The task deals with Multimedia Automatic Misogyny Identification (MAMI).Hateful memes need to be detected within a data collection.For this task, we implemented six models for text and image analysis and tested the effectiveness of their combinations.A fusion system implements a multi-modal transformer to integrate the embeddings of these models.The best performing models included BERT for the text of the meme, manually derived associations for words in the memes and a Faster R-CNN network for the image.We evaluated the performance of our approach also with the data of the Facebook Hateful Memes challenge in order to analyze the generalisation capabilities of the approach.
MT-Speech at SemEval-2022 Task 10: Incorporating Data Augmentation and Auxiliary Task with Cross-Lingual Pretrained Language Model for Structured Sentiment Analysis

Structured Sentiment Analysis (SSA) is an important component of sentiment analysis, which is a critical task in NLP.Traditional SSA methods cannot capture the cross-lingual interactions between different language and there is insufficient annotated data, especially in the cross-lingual settings.In this paper, we use the pre-trained language models with two auxiliary tasks and adopt data augmentation to address the above problems.Specifically, we employ XLM-RoBERTa to capture the cross-lingual knowledge interactions and enhance the generalization in multilingual/cross-lingual settings.Furthermore, we leverage two data augmentation techniques and propose two auxiliary tasks to improve the performance on the few-shot and zero-shot settings.Experiments demonstrate that our model ranks first on the cross-lingual sub-task and second on the monolingual subtask of SemEval-2022 task 10.
Exposing the Limits of Video-Text Models through Contrast Sets

Recent video-text models can retrieve relevant videos based on text with a high accuracy, but to what extent do they comprehend the semantics of the text?Can they discriminate between similar entities and actions?To answer this, we propose an evaluation framework that probes video-text models with hard negatives.We automatically build contrast sets, where true textual descriptions are manipulated in ways that change their semantics while maintaining plausibility.Specifically, we leverage a pre-trained language model and a set of heuristics to create verb and person entity focused contrast sets.We apply these in the multiple choice video-totext classification setting.We test the robustness of recent methods on the proposed automatic contrast sets, and compare them to additionally collected human-generated counterparts, to assess their effectiveness.We see that model performance suffers across all methods, erasing the gap between recent CLIP-based methods vs. the earlier methods. 1
PALI-NLP at SemEval-2022 Task 4: Discriminative Fine-tuning of Transformers for Patronizing and Condescending Language Detection

Patronizing and condescending language (PCL) has a large harmful impact and is difficult to detect, both for human judges and existing NLP systems.At SemEval-2022 Task 4, we propose a novel Transformer-based model and its ensembles to accurately understand such language context for PCL detection.To facilitate comprehension of the subtle and subjective nature of PCL, two fine-tuning strategies are applied to capture discriminative features from diverse linguistic behaviour and categorical distribution.The system achieves remarkable results on the official ranking, including 1st in Subtask 1 and 5th in Subtask 2. Extensive experiments on the task demonstrate the effectiveness of our system and its strategies.
Identifying Implicitly Abusive Remarks about Identity Groups using a Linguistically Informed Approach

We address the task of distinguishing implicitly abusive sentences on identity groups (Muslims terrorize the world daily) from other grouprelated negative polar sentences (Muslims despise terrorism).Implicitly abusive language are utterances not conveyed by abusive words (e.g.bimbo or scum).So far, the detection of such utterances could not be properly addressed since existing datasets displaying a high degree of implicit abuse are fairly biased.Following the recently proposed strategy to solve implicit abuse by separately addressing its different subtypes, we present a new focused and less biased dataset that consists of the subtype of atomic negative sentences about identity groups.For that task, we model components that each address one facet of such implicit abuse, i.e. depiction as perpetrators, aspectual classification and non-conformist views.The approach generalizes across different identity groups and languages.
Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models

The energy requirements of current natural language processing models continue to grow at a rapid, unsustainable pace.Recent works highlighting this problem conclude there is an urgent need for methods that reduce the energy needs of NLP and machine learning more broadly.In this article, we investigate techniques that can be used to reduce the energy consumption of common NLP applications.In particular, we focus on techniques to measure energy usage and different hardware and datacenter-oriented settings that can be tuned to reduce energy consumption for training and inference for language models.We characterize the impact of these settings on metrics such as computational performance and energy consumption through experiments conducted on a high performance computing system as well as popular cloud computing platforms.These techniques can lead to significant reduction in energy consumption when training language models or their use for inference.For example, power-capping, which limits the maximum power a GPU can consume, can enable a 15% decrease in energy usage with marginal increase in overall computation time when training a transformer-based language model. 1
Training Mixed-Domain Translation Models via Federated Learning

Peyman Passban, Tanya Roosta, Rahul Gupta, Ankit Chadha, Clement Chung. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.
Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts

Fine-tuning continuous prompts for target tasks has recently emerged as a compact alternative to full model fine-tuning.Motivated by these promising results, we investigate the feasibility of extracting a discrete (textual) interpretation of continuous prompts that is faithful to the problem they solve.In practice, we observe a "wayward" behavior between the task solved by continuous prompts and the nearest neighbor discrete projections of these prompts: One can find continuous prompts that solve a task while being projected to an arbitrary text (e.g., definition of a different or even a contradictory task) and simultaneously being within a very small (2%) margin of the best continuous prompt of the same size for the task.We provide intuitions behind this odd and surprising behavior, as well as extensive empirical analyses quantifying the effect of design choices.For instance, larger models exhibit higher waywardness, i.e, we can find prompts that more closely map to any arbitrary text with a smaller drop of accuracy.These findings have important implications relating to the difficulty of faithfully interpreting continuous prompts and their generalization across models and tasks, providing guidance for future progress in prompting language models.
An Incremental Interesting Maximal Frequent Itemset Mining Based on FP-Growth Algorithm

Frequent itemset mining is the most important step of association rule mining. It plays a very important role in incremental data environments. The massive volume of data creates an imminent need to design incremental algorithms for the maximal frequent itemset mining in order to handle incremental data over time. In this study, we propose an incremental maximal frequent itemset mining algorithms that integrate subjective interestingness criterion during the process of mining. The proposed framework is designed to deal with incremental data, which usually come at different times. It extends FP-Max algorithm, which is based on FP-Growth method by pushing interesting measures during maximal frequent itemset mining, and performs dynamic and early pruning to leave uninteresting frequent itemsets in order to avoid uninteresting rule generation. The framework was implemented and tested on public databases, and the results found are promising.
AI-Driven Performance Modeling for AI Inference Workloads

Deep Learning (DL) is moving towards deploying workloads not only in cloud datacenters, but also to the local devices. Although these are mostly limited to inference tasks, it still widens the range of possible target architectures significantly. Additionally, these new targets usually come with drastically reduced computation performance and memory sizes compared to the traditionally used architectures—and put the key optimization focus on the efficiency as they often depend on batteries. To help developers quickly estimate the performance of a neural network during its design phase, performance models could be used. However, these models are expensive to implement as they require in-depth knowledge about the hardware architecture and the used algorithms. Although AI-based solutions exist, these either require large datasets that are difficult to collect on the low-performance targets and/or limited to a small number of target platforms and metrics. Our solution exploits the block-based structure of neural networks, as well as the high similarity in the typically used layer configurations across neural networks, enabling the training of accurate models on significantly smaller datasets. In addition, our solution is not limited to a specific architecture or metric. We showcase the feasibility of the solution on a set of seven devices from four different hardware architectures, and with up to three performance metrics per target—including the power consumption and memory footprint. Our tests have shown that the solution achieved an error of less than 1 ms (2.6%) in latency, 0.12 J (4%) in energy consumption and 11 MiB (1.5%) in memory allocation for the whole network inference prediction, while being up to five orders of magnitude faster than a benchmark.
Compound Conditionals as Random Quantities and Boolean Algebras

Conditionals play a key role in different areas of logic and probabilistic reasoning, and they have been studied and formalised from different angles. In this paper we focus on the de Finetti's notion of conditional as a three-valued object, with betting-based semantics, and its related approach as random quantity as mainly developed by two of the authors. Compound conditionals have been studied in the literature, but not in full generality. In this paper we provide a natural procedure to explicitly attach conditional random quantities to arbitrary compound conditionals that also allows us to compute their previsions. By studying the properties of these random quantities, we show that, in fact, the set of compound conditionals can be endowed with a Boolean algebraic structure. In doing so, we pave the way to build a bridge between the long standing tradition of three-valued conditionals and a more recent proposal of looking at conditionals as elements from suitable Boolean algebras.
A Comparative Study of Multiple Deep Learning Models Based on Multi-Input Resolution for Breast Ultrasound Images

The purpose of this study was to explore the performance of different parameter combinations of deep learning (DL) models (Xception, DenseNet121, MobileNet, ResNet50 and EfficientNetB0) and input image resolutions (REZs) (224 × 224, 320 × 320 and 488 × 488 pixels) for breast cancer diagnosis.This multicenter study retrospectively studied gray-scale ultrasound breast images enrolled from two Chinese hospitals. The data are divided into training, validation, internal testing and external testing set. Three-hundreds images were randomly selected for the physician-AI comparison. The Wilcoxon test was used to compare the diagnose error of physicians and models under P=0.05 and 0.10 significance level. The specificity, sensitivity, accuracy, area under the curve (AUC) were used as primary evaluation metrics.A total of 13,684 images of 3447 female patients are finally included. In external test the 224 and 320 REZ achieve the best performance in MobileNet and EfficientNetB0 respectively (AUC: 0.893 and 0.907). Meanwhile, 448 REZ achieve the best performance in Xception, DenseNet121 and ResNet50 (AUC: 0.900, 0.883 and 0.871 respectively). In physician-AI test set, the 320 REZ for EfficientNetB0 (AUC: 0.896, P < 0.1) is better than senior physicians. Besides, the 224 REZ for MobileNet (AUC: 0.878, P < 0.1), 448 REZ for Xception (AUC: 0.895, P < 0.1) are better than junior physicians. While the 448 REZ for DenseNet121 (AUC: 0.880, P < 0.05) and ResNet50 (AUC: 0.838, P < 0.05) are only better than entry physicians.Based on the gray-scale ultrasound breast images, we obtained the best DL combination which was better than the physicians.