Reviewerly/exHarmony huggingface dataset:
Attention as Grounding: Exploring Textual and Cross-Modal Attention on Entities and Relations in Language-and-Vision Transformer

We explore how a multi-modal transformer trained for generation of longer image descriptions learns syntactic and semantic representations about entities and relations grounded in objects at the level of masked self-attention (text generation) and crossmodal attention (information fusion).We observe that cross-attention learns the visual grounding of noun phrases into objects and high-level semantic information about spatial relations, while text-to-text attention captures low-level syntactic knowledge between words.This concludes that language models in a multi-modal task learn different semantic information about objects and relations cross-modally and uni-modally (text-only).
Knowledge-Augmented Methods for Natural Language Processing

Knowledge in natural language processing (NLP) has been a rising trend especially after the advent of large scale pre-trained models. NLP models with attention to knowledge can i) access unlimited amount of external information; ii) delegate the task of storing knowledge from its parameter space to knowledge sources; iii) obtain up-to-date information; iv) make prediction results more explainable via selected knowledge. In this tutorial, we will introduce the key steps in integrating knowledge into NLP, including knowledge grounding from text, knowledge representation and fusing. In addition, we will introduce recent state-of-the-art applications in fusing knowledge into language understanding, language generation and commonsense reasoning.
Answer-level Calibration for Free-form Multiple Choice Question Answering

Pre-trained language models have recently shown that training on large corpora using the language modeling objective enables few-shot and zero-shot capabilities on a variety of NLP tasks, including commonsense reasoning tasks.This is achieved using text interactions with the model, usually by posing the task as a natural language text completion problem.While using language model probabilities to obtain task specific scores has been generally useful, it often requires task-specific heuristics such as length normalization, or probability calibration.In this work, we consider the question answering format, where we need to choose from a set of (free-form) textual choices of unspecified lengths given a context.We present ALC (Answer-Level Calibration), where our main suggestion is to model context-independent biases in terms of the probability of a choice without the associated context and to subsequently remove it using an unsupervised estimate of similarity with the full context.We show that our unsupervised answer-level calibration consistently improves over or is competitive with baselines using standard evaluation metrics on a variety of tasks including commonsense reasoning tasks.Further, we show that popular datasets potentially favor models biased towards easy cues which are available independent of the context.We analyze such biases using an associated F1-score.Our analysis indicates that answer-level calibration is able to remove such biases and leads to a more robust measure of model capability.
On the Politics of Design Framing Practices

Abstract In this conceptual article, our aim is to deconstruct the conceptualization of design framing and establish its essentially political nature. It demonstrates the positionality inherent within frames insofar as frames articulate subordinated or dominant status, or express normative understandings until challenged. In doing so, we build a conceptualization of the political foundations of design framing practices and their implications for those contexts within which design operates. Consequently, we argue for dissensual counter-framing design practices that unsettle institutionalized norms and ideologies played out within frames, and through which a form of political agency is sociomaterially enacted.
An in-Depth Analysis of the Software Features’ Impact on the Performance of Deep Learning-Based Software Defect Predictors

Software Defects Prediction represents an essential activity during software development that contributes to continuously improving software quality and software maintenance and evolution by detecting defect-prone modules in new versions of a software system.In this paper, we are conducting an in-depth analysis on the software features' impact on the performance of deep learning-based software defect predictors.We further extend a large-scale feature set proposed in the literature for detecting defect-proneness, by adding conceptual software features that capture the semantics of the source code, including comments.The conceptual features are automatically engineered using Doc2Vec, an artificial neural network based prediction model.A broad evaluation performed on the Calcite software system highlights a statistically significant improvement obtained by applying deep learning-based classifiers for detecting software defects when using conceptual features extracted from the source code for characterizing the software entities.
Alignment-Augmented Consistent Translation for Multilingual Open Information Extraction

Progress with supervised Open Information Extraction (OpenIE) has been primarily limited to English due to the scarcity of training data in other languages.In this paper, we explore techniques to automatically convert English text for training OpenIE systems in other languages.We introduce the Alignment-Augmented Consistent Translation (AACTRANS) model to translate English sentences and their corresponding extractions consistently with each other -with no changes to vocabulary or semantic meaning which may result from independent translations.Using the data generated with AACTRANS, we train a novel two-stage generative OpenIE model, which we call GEN2OIE, that outputs for each sentence: 1) relations in the first stage and 2) all extractions containing the relation in the second stage.GEN2OIE increases relation coverage using a training data transformation technique that is generalizable to multiple languages, in contrast to existing models that use an English-specific training loss.Evaluations on 5 languages -Spanish, Portuguese, Chinese, Hindi and Telugu -show that the GEN2OIE with AACTRANS data outperforms prior systems by a margin of 6-25% F1. 1
Your Answer is Incorrect... Would you like to know why? Introducing a Bilingual Short Answer Feedback Dataset

Handing in a paper or exercise and merely receiving "bad" or "incorrect" as feedback is not very helpful when the goal is to improve.Unfortunately, this is currently the kind of feedback given by many Automatic Short Answer Grading (ASAG) systems.One of the reasons for this is a lack of content-focused elaborated feedback datasets.To encourage research on explainable and understandable feedback systems, we present the Short Answer Feedback dataset (SAF).Similar to other ASAG datasets, SAF contains learner responses and reference answers to German and English questions.However, instead of only assigning a label or score to the learners' answers, SAF also contains elaborated feedback explaining the given score.Thus, SAF enables supervised training of models that grade answers and explain where and why mistakes were made.This paper discusses the need for enhanced feedback models in real-world pedagogical scenarios, describes the dataset annotation process, gives a comprehensive analysis of SAF, and provides T5-based baselines for future comparison. 1
Blockchain-Based Transaction Manager for Ontology Databases

Knowledge graphs are commonly represented by ontology-based databases. Tracking the provenance of ontological changes and ensuring ontology consistency is important. In this work, we propose a transaction manager for ontology-based database manipulation that combines blockchain and Semantic Web technologies. The latter is used for the efficient querying and modification of data, whereas the blockchain is used for the secure storage and tracking of changes. The blockchain enables a decentralized setup and data restoration. We evaluate our solution by measuring cost and time. Our solution introduces some overhead for updates whereas querying works at the same speed as the underlying ontology database.
Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features

While neural text-to-speech systems perform remarkably well in high-resource scenarios, they cannot be applied to the majority of the over 6,000 spoken languages in the world due to a lack of appropriate training data.In this work, we use embeddings derived from articulatory vectors rather than embeddings derived from phoneme identities to learn phoneme representations that hold across languages.In conjunction with language agnostic meta learning, this enables us to fine-tune a high-quality textto-speech model on just 30 minutes of data in a previously unseen language spoken by a previously unseen speaker.
Probing for Predicate Argument Structures in Pretrained Language Models

Thanks to the effectiveness and wide availability of modern pretrained language models (PLMs), recently proposed approaches have achieved remarkable results in dependencyand span-based, multilingual and cross-lingual Semantic Role Labeling (SRL).These results have prompted researchers to investigate the inner workings of modern PLMs with the aim of understanding how, where, and to what extent they encode information about SRL.In this paper, we follow this line of research and probe for predicate argument structures in PLMs.Our study shows that PLMs do encode semantic structures directly into the contextualized representation of a predicate, and also provides insights into the correlation between predicate senses and their structures, the degree of transferability between nominal and verbal structures, and how such structures are encoded across languages.Finally, we look at the practical implications of such insights and demonstrate the benefits of embedding predicate argument structure information into an SRL model.
DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations

Automatic evaluation metrics are essential for the rapid development of open-domain dialogue systems as they facilitate hyperparameter tuning and comparison between models.Although recently proposed trainable conversation-level metrics have shown encouraging results, the quality of the metrics is strongly dependent on the quality of training data.Prior works mainly resort to heuristic textlevel manipulations (e.g.utterances shuffling) to bootstrap incoherent conversations (negative examples) from coherent dialogues (positive examples). Such approaches are insufficient to appropriately reflect the incoherence that occurs in interactions between advanced dialogue models and humans.To tackle this problem, we propose DEAM, a Dialogue coherence Evaluation metric that relies on Abstract Meaning Representation (AMR) to apply semanticlevel Manipulations for incoherent (negative) data generation.AMRs naturally facilitate the injection of various types of incoherence sources, such as coreference inconsistency, irrelevancy, contradictions, and decrease engagement, at the semantic level, thus resulting in more natural incoherent samples.Our experiments show that DEAM 1 achieves higher correlations with human judgments compared to baseline methods on several dialog datasets by significant margins.We also show that DEAM can distinguish between coherent and incoherent dialogues generated by baseline manipulations, whereas those baseline models cannot detect incoherent examples generated by DEAM.Our results demonstrate the potential of AMRbased semantic manipulations for natural negative example generation.
Overview of Abusive Comment Detection in Tamil-ACL 2022

The social media is one of the significant digital platforms that create a huge impact in peoples of all levels.The comments posted on social media is powerful enough to even change the political and business scenarios in very few hours.They also tend to attack a particular individual or a group of individuals.This shared task aims at detecting the abusive comments involving, Homophobia, Misandry, Counterspeech, Misogyny, Xenophobia, Transphobic.The hope speech is also identified.A dataset collected from social media tagged with the above said categories in Tamil and Tamil-English code-mixed languages are given to the participants.The participants used different machine learning and deep learning algorithms.This paper presents the overview of this task comprising the dataset details and results of the participants.
Adjusting the Precision-Recall Trade-Off with Align-and-Predict Decoding for Grammatical Error Correction

Modern writing assistance applications are always equipped with a Grammatical Error Correction (GEC) model to correct errors in user-entered sentences.Different scenarios have varying requirements for correction behavior, e.g., performing more precise corrections (high precision) or providing more candidates for users (high recall).However, previous works adjust such trade-off only for sequence labeling approaches.In this paper, we propose a simple yet effective counterpart -Align-and-Predict Decoding (APD) for the most popular sequence-to-sequence models to offer more flexibility for the precision-recall trade-off.During inference, APD aligns the already generated sequence with input and adjusts scores of the following tokens.Experiments in both English and Chinese GEC benchmarks show that our approach not only adapts a single model to precision-oriented and recall-oriented inference, but also maximizes its potential to achieve state-of-the-art results.Our code is available at https://github.com/AutoTemp/Align-and-Predict.
Overview of the MedVidQA 2022 Shared Task on Medical Video Question-Answering

In this paper, we present an overview of the MedVidQA 2022 shared task, collocated with the 21st BioNLP workshop at ACL 2022.The shared task addressed two of the challenges faced by medical video question answering: (i) a video classification task that explores new approaches to medical video understanding (labeling), and (ii) a visual answer localization task.Visual answer localization refers to identification of the relevant temporal segments (start and end timestamps) in the video where the answer to the medical question is being shown or illustrated.A total of thirteen teams participated in the shared task challenges, with eleven system descriptions submitted to the workshop.The descriptions present monomodal and multi-modal approaches developed for medical video classification and visual answer localization.This paper describes the tasks, the datasets, evaluation metrics, and baseline systems for both tasks.Finally, the paper summarizes the techniques and results of the evaluation of the various approaches explored by the participating teams.
Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model

In this paper, we describe a BERT model trained on the Eighteenth Century Collections Online (ECCO) dataset of digitized documents.The ECCO dataset poses unique modelling challenges due to the presence of Optical Character Recognition (OCR) artifacts.We establish the performance of the BERT model on a publication year prediction task against linear baseline models and human judgement, finding the BERT model to be superior to both and able to date the works, on average, with less than 7 years absolute error.We also explore how language change over time affects the model by analyzing the features the model uses for publication year predictions as given by the Integrated Gradients model explanation method.
MMCoQA: Conversational Question Answering over Text, Tables, and Images

The rapid development of conversational assistants accelerates the study on conversational question answering (QA).However, the existing conversational QA systems usually answer users' questions with a single knowledge source, e.g., paragraphs or a knowledge graph, but overlook the important visual cues, let alone multiple knowledge sources of different modalities.In this paper, we hence define a novel research task, i.e., multimodal conversational question answering (MMCoQA), aiming to answer users' questions with multimodal knowledge sources via multi-turn conversations.This new task brings a series of research challenges, including but not limited to priority, consistency, and complementarity of multimodal knowledge.To facilitate the data-driven approaches in this area, we construct the first multimodal conversational QA dataset, named MMConvQA.Questions are fully annotated with not only natural language answers but also the corresponding evidence and valuable decontextualized self-contained questions.Meanwhile, we introduce an endto-end baseline model, which divides this complex research task into question understanding, multi-modal evidence retrieval, and answer extraction.Moreover, we report a set of benchmarking results, and the results indicate that there is ample room for improvement.Which city features a green copper statue of a woman holding a torch?New York City.What is the largest catholic church in it?
Distributed Ledger Technology and Climate Finance

Enormous amounts of capital will need to be mobilized in the coming decades to drive the transition of our global economy toward climate neutrality. In recent years, the financial system started to move in a more environmentally sustainable direction, inter alia offering more green investment opportunities. Speeding up this process is inevitable if humanity wants to meet the Sustainable Development Goals (SDGs) and achieve the targets of the Paris Climate Agreement. Distributed Ledger Technology (DLT) may enable climate finance to effectively accelerate this transition toward sustainability. Inherent characteristics like security, immutability, transparency, and auditability make DLT a highly promising tool for climate-related use cases. However, the mainstream adoption of these digital innovations is still constrained by various obstacles, including technological and regulatory barriers, especially in the highly regulated finance sector. Existing digital divides between developed and developing nations further exacerbate these challenges. Based on these initial deliberations, this chapter shows how DLT can be used effectively for innovative climate finance. We first introduce the technology and discuss its sustainability as well as different possible DLT governance structures. Based on this introduction, we provide an overview of sustainability-related initiatives and networks linked to DLT. We review various DLT applications in Decentralized Finance (DeFi), asset management, Measurement, Reporting and Verification (MRV), tokenization, and other relevant fields. We conclude with an outlook on the role of DLT in climate finance and present recommendations for overcoming some of the remaining risks and obstacles.
Probing as Quantifying Inductive Bias

Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks.Such performance improvements have motivated researchers to quantify and understand the linguistic information encoded in these representations.In general, researchers quantify the amount of linguistic information through probing, an endeavor which consists of training a supervised model to predict a linguistic property directly from the contextual representations.Unfortunately, this definition of probing has been subject to extensive criticism in the literature, and has been observed to lead to paradoxical and counterintuitive results.In the theoretical portion of this paper, we take the position that the goal of probing ought to be measuring the amount of inductive bias that the representations encode on a specific task.We further describe a Bayesian framework that operationalizes this goal and allows us to quantify the representations' inductive bias.In the empirical portion of the paper, we apply our framework to a variety of NLP tasks.Our results suggest that our proposed framework alleviates many previous problems found in probing.Moreover, we are able to offer concrete evidence that-for some tasks-fastText can offer a better inductive bias than BERT.
Joint Entity and Relation Extraction Based on Table Labeling Using Convolutional Neural Networks

This study introduces a novel approach to the joint extraction of entities and relations by stacking convolutional neural networks (CNNs) on pretrained language models.We adopt table representations to model the entities and relations, casting the entity and relation extraction as a table-labeling problem.Regarding each table as an image and each cell in a table as an image pixel, we apply two-dimensional CNNs to the tables to capture local dependencies and predict the cell labels.The experimental results showed that the performance of the proposed method is comparable to those of current state-of-art systems on the CoNLL04, ACE05, and ADE datasets.Even when freezing pretrained language model parameters, the proposed method showed a stable performance, whereas the compared methods suffered from significant decreases in performance.This observation indicates that the parameters of the pretrained encoder may incorporate dependencies among the entity and relation labels during fine-tuning.B Effect of Parameter Updates with BERT (cont.)
PICT@DravidianLangTech-ACL2022: Neural Machine Translation On Dravidian Languages

This paper presents a summary of the findings that we obtained based on the shared task on machine translation of Dravidian languages.We stood first in three of the five sub-tasks which were assigned to us for the main shared task.We carried out neural machine translation for the following five language pairs: Kannada to Tamil, Kannada to Telugu, Kannada to Malayalam, Kannada to Sanskrit, and Kannada to Tulu.The datasets for each of the five language pairs were used to train various translation models, including Seq2Seq models such as LSTM, bidirectional LSTM, Conv2Seq, and training state-of-the-art as transformers from scratch, and fine-tuning already pre-trained models.For some models involving monolingual corpora, we implemented backtranslation as well.These models' accuracy was later tested with a part of the same dataset using BLEU score as an evaluation metric.
CANFIS: A Chaos Adaptive Neural Fuzzy Inference System for Workload Prediction in the Cloud

As cloud-based applications become increasingly solicited by companies and individuals, the competition between cloud providers that offer cloud services keeps increasing.To win this competition, cloud providers must provide sufficient computing resources that will satisfy users' demand for every request.Workload prediction has been investigated extensively to resolve this issue using various techniques.This paper presents a workload prediction method called CANFIS, which combines the Savitzky-Golay (SG) filter and Chaotic time series analysis with the Adaptive Neural Fuzzy Inference System (ANFIS) to make predictions of cloud workloads.The SG filter is used to clean data from noise and outliers, and chaotic analysis is used to investigate the chaotic nature of workload and to build the improved ANFIS model.The proposed method is evaluated using real workload traces from web applications (i.e., Wikipedia and NASA Kennedy traces) and cluster applications (i.e., CPU and Memory of Google cluster).Experimental results show that the proposed CANFIS model can improve prediction accuracy compared to existing techniques, including simple ANFIS, Auto-Regressive Integrated Moving Average (ARIMA), Support Vector Machine (SVM), Long Short Term Memory (LSTM), and Neural Networks based methods.A statistical analysis is also performed using the Friedman test, along with Finner post hoc analysis to verify the efficiency of the proposed prediction model.