Reviewerly/exHarmony huggingface dataset:
Program Synthesis for Complex QA on Charts via Probabilistic Grammar Based Filtered Iterative Back-Translation

Answering complex reasoning questions from chart images is a challenging problem requiring a combination of natural language understanding, fine-grained perception, and analytical reasoning. Current chart-based Question Answering (QA) approaches largely address structural, visual or simple data retrieval-type questions with fixed-vocabulary answers and perform poorly on reasoning queries. We focus on answering realistic, complex, reasoning-based questions where the answer needs to be computed and not selected from a fixed set of choices. Our approach employs a neural semantic parser to transform Natural Language (NL) questions into SQL programs and execute them on a standardized schema populated from the extracted chart contents. In the absence of program annotations, i.e., in a weak supervision setting, we obtain initial SQL predictions from a pre-trained CodeT5 semantic parser and employ Filtered Iterative Back-Translation (FIBT) for iteratively augmenting our NL-SQL training set. The forward (neural semantic parser) and backward (language model) models are initially trained with an external NL-SQL dataset. We iteratively move towards the NL query distribution by generating NL questions from the synthesized SQL programs using a Probabilistic Context-Free Grammar (PCFG) where the production rule probabilities are induced to be inversely proportional to the probabilities in the training data. We filter out the generated NL queries with mismatched structures and compositions. Our FIBT approach achieves State-of-the-Art (SOTA) results on reasoning-based queries in the PlotQA dataset yielding a test accuracy of 60.44%, superseding the previous baselines by a large margin.
Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis

Recent advances in the area of long document matching have primarily focused on using transformer-based models for long document encoding and matching. There are two primary challenges associated with these models. Firstly, the performance gain provided by transformer-based models comes at a steep cost – both in terms of the required training time and the resource (memory and energy) consumption. The second major limitation is their inability to handle more than a pre-defined input token length at a time. In this work, we empirically demonstrate the effectiveness of simple neural models (such as feed-forward networks, and CNNs) and simple embeddings (like GloVe, and Paragraph Vector) over transformer-based models on the task of document matching. We show that simple models outperform the more complex BERT-based models while taking significantly less training time, energy, and memory. The simple models are also more robust to variations in document length and text perturbations.
Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronouns

Bias-measuring datasets play a critical role in detecting biased behavior of language models and in evaluating progress of bias mitigation methods. In this work, we focus on evaluating gender bias through coreference resolution, where previous datasets are either hand-crafted or fail to reliably measure an explicitly defined bias. To overcome these shortcomings, we propose a novel method to collect diverse, natural, and minimally distant text pairs via counterfactual generation, and construct Counter-GAP, an annotated dataset consisting of 4008 instances grouped into 1002 quadruples. We further identify a bias cancellation problem in previous group-level metrics on Counter-GAP, and propose to use the difference between inconsistency across genders and within genders to measure bias at a quadruple level. Our results show that four pre-trained language models are significantly more inconsistent across different gender groups than within each group, and that a name-based counterfactual data augmentation method is more effective to mitigate such bias than an anonymization-based method.
FVQA 2.0: Introducing Adversarial Samples into Fact-based Visual Question Answering

The widely used Fact-based Visual Question Answering (FVQA) dataset contains visually-grounded questions that require information retrieval using common sense knowledge graphs to answer. It has been observed that the original dataset is highly imbalanced and concentrated on a small portion of its associated knowledge graph. We introduce FVQA 2.0 which contains adversarial variants of test questions to address this imbalance. We show that systems trained with the original FVQA train sets can be vulnerable to adversarial samples and we demonstrate an augmentation scheme to reduce this vulnerability without human annotations.
Extracting Victim Counts from Text

Decision-makers in the humanitarian sector rely on timely and exact information during crisis events. Knowing how many civilians were injured during an earthquake is vital to allocate aids properly. Information about such victim counts are however often only available within full-text event descriptions from newspapers and other reports. Extracting numbers from text is challenging: numbers have different formats and may require numeric reasoning. This renders purely tagging approaches insufficient. As a consequence, fine-grained counts of injured, displaced, or abused victims beyond fatalities are often not extracted and remain unseen. We cast victim count extraction as a question answering (QA) task with a regression or classification objective. We compare tagging approaches: regex, dependency parsing, semantic role labeling, and advanced text-to-text models. Beyond model accuracy, we analyze extraction reliability and robustness which are key for this sensitive task. In particular, we discuss model calibration and investigate out-of-distribution and few-shot performance. Ultimately, we make a comprehensive recommendation on which model to select for different desiderata and data domains. Our work is among the first to apply numeracy-focused large language models in a real-world use case with a positive impact.
ComSearch: Equation Searching with Combinatorial Strategy for Solving Math Word Problems with Weak Supervision

Previous studies have introduced a weakly-supervised paradigm for solving math word problems requiring only the answer value annotation. While these methods search for correct value equation candidates as pseudo labels, they search among a narrow sub-space of the enormous equation space. To address this problem, we propose a novel search algorithm with combinatorial strategy ComSearch, which can compress the search space by excluding mathematically equivalent equations. The compression allows the searching algorithm to enumerate all possible equations and obtain high-quality data. We investigate the noise in the pseudo labels that hold wrong mathematical logic, which we refer to as the false-matching problem, and propose a ranking model to denoise the pseudo labels. Our approach holds a flexible framework to utilize two existing supervised math word problem solvers to train pseudo labels, and both achieve state-of-the-art performance in the weak supervision task.
Unsupervised Anomaly Detection in Multi-Topic Short-Text Corpora

Unsupervised anomaly detection seeks to identify deviant data samples in a dataset without using labels and constitutes a challenging task, particularly when the majority class is heterogeneous. This paper addresses this topic for textual data and aims to determine whether a text sample is an outlier within a potentially multi-topic corpus. To this end, it is crucial to grasp the semantic aspects of words, particularly when dealing with short texts, since it is difficult to syntactically discriminate data samples based only on a few words. Thereby we make use of word embeddings to represent each sample by a dense vector, efficiently capturing the underlying semantics. Then, we rely on the Mixture Model approach to detect which samples deviate the most from the underlying distributions of the corpus. Experiments carried out on real datasets show the effectiveness of the proposed approach in comparison to state-of-the-art techniques both in terms of performance and time efficiency, especially when more than one topic is present in the corpus.
DiffQG: Generating Questions to Summarize Factual Changes

Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos, Michael J.Q. Zhang, Eunsol Choi, Bhuwan Dhingra. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.
Towards a Unified Multi-Domain Multilingual Named Entity Recognition Model

Mayank Kulkarni, Daniel Preotiuc-Pietro, Karthik Radhakrishnan, Genta Indra Winata, Shijie Wu, Lingjue Xie, Shaohua Yang. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.
Multimodal Graph Transformer for Multimodal Question Answering

Despite the success of Transformer models in vision and language tasks, they often learn knowledge from enormous data implicitly and cannot utilize structured input data directly. On the other hand, structured learning approaches such as graph neural networks (GNNs) that integrate prior information can barely compete with Transformer models. In this work, we aim to benefit from both worlds and propose a novel Multimodal Graph Transformer for question answering tasks that requires performing reasoning across multiple modalities. We introduce a graph-involved plug-and-play quasi-attention mechanism to incorporate multimodal graph information, acquired from text and visual data, to the vanilla self-attention as effective prior. In particular, we construct the text graph, dense region graph, and semantic graph to generate adjacency matrices, and then compose them with input vision and language features to perform downstream reasoning. Such a way of regularizing self-attention with graph information significantly improves the inferring ability and helps align features from different modalities. We validate the effectiveness of Multimodal Graph Transformer over its Transformer baselines on GQA, VQAv2, and MultiModalQA datasets.
Do we need Label Regularization to Fine-tune Pre-trained Language Models?

Ivan Kobyzev, Aref Jafari, Mehdi Rezagholizadeh, Tianda Li, Alan Do-Omri, Peng Lu, Pascal Poupart, Ali Ghodsi. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.
Text-Blueprint: An Interactive Platform for Plan-based Conditional Generation

Fantine Huot, Joshua Maynez, Shashi Narayan, Reinald Kim Amplayo, Kuzman Ganchev, Annie Priyadarshini Louis, Anders Sandholm, Dipanjan Das, Mirella Lapata. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. 2023.
Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies

Prior studies in privacy policies frame the question answering (QA) task as identifying the most relevant text segment or a list of sentences from a policy document given a user query. Existing labeled datasets are heavily imbalanced (only a few relevant segments), limiting the QA performance in this domain. In this paper, we develop a data augmentation framework based on ensembling retriever models that captures the relevant text segments from unlabeled policy documents and expand the positive examples in the training set. In addition, to improve the diversity and quality of the augmented data, we leverage multiple pre-trained language models (LMs) and cascaded them with noise reduction oracles. Using our augmented data on the PrivacyQA benchmark, we elevate the existing baseline by a large margin (10% F1) and achieve a new state-of-the-art F1 score of 50%. Our ablation studies provide further insights into the effectiveness of our approach.
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer

Shanu Kumar, Soujanya Abbaraju, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.
Self-imitation Learning for Action Generation in Text-based Games

In this work, we study reinforcement learning (RL) in solving text-based games. We address the challenge of combinatorial action space, by proposing a confidence-based self-imitation model to generate action candidates for the RL agent. Firstly, we leverage the self-imitation learning to rank and exploit past valuable trajectories to adapt a pre-trained language model (LM) towards a target game. Then, we devise a confidence-based strategy to measure the LM’s confidence with respect to a state, thus adaptively pruning the generated actions to yield a more compact set of action candidates. In multiple challenging games, our model demonstrates promising performance in comparison to the baselines.
PiC: A Phrase-in-Context Dataset for Phrase Understanding and Semantic Search

While contextualized word embeddings have been a de-facto standard, learning contextualized phrase embeddings is less explored and being hindered by the lack of a human-annotated benchmark that tests machine understanding of phrase semantics given a context sentence or paragraph (instead of phrases alone). To fill this gap, we propose PiC—a dataset of ∼28K of noun phrases accompanied by their contextual Wikipedia pages and a suite of three tasks for training and evaluating phrase embeddings. Training on PiC improves ranking-models’ accuracy and remarkably pushes span selection (SS) models (i.e., predicting the start and end index of the target phrase) near human accuracy, which is 95% Exact Match (EM) on semantic search given a query phrase and a passage. Interestingly, we find evidence that such impressive performance is because the SS models learn to better capture the common meaning of a phrase regardless of its actual context. SotA models perform poorly in distinguishing two senses of the same phrase in two contexts (∼60% EM) and in estimating the similarity between two different phrases in the same context (∼70% EM).
PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation

Curriculum Data Augmentation (CDA) improves neural models by presenting synthetic data with increasing difficulties from easy to hard. However, traditional CDA simply treats the ratio of word perturbation as the difficulty measure and goes through the curriculums only once. This paper presents PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation, a novel CDA framework via paraphrasing, which exploits the textual paraphrase similarity as the curriculum difficulty measure. We propose a curriculum-aware paraphrase generation module composed of three units: a paraphrase candidate generator with bottom-k sampling, a filtering mechanism and a difficulty measure. We also propose a cyclic learning strategy that passes through the curriculums multiple times. The bottom-k sampling is proposed to generate super-hard instances for the later curriculums. Experimental results on few-shot text classification as well as dialogue generation indicate that PCC surpasses competitive baselines. Human evaluation and extensive case studies indicate that bottom-k sampling effectively generates super-hard instances, and PCC significantly improves the baseline dialogue agent.
Patch seriation to visualize data and model parameters

Abstract We developed a new seriation merit function for enhancing the visual information of data matrices. A local similarity matrix is calculated, where the average similarity of neighbouring objects is calculated in a limited variable space and a global function is constructed to maximize the local similarities and cluster them into patches by simple row and column ordering. The method identifies data clusters in a powerful way, if the similarity of objects is caused by some variables and these variables differ for the distinct clusters. The method can be used in the presence of missing data and also on more than two-dimensional data arrays. We show the feasibility of the method on different data sets: on QSAR, chemical, material science, food science, cheminformatics and environmental data in two- and three-dimensional cases. The method can be used during the development and the interpretation of artificial neural network models by seriating different features of the models. It helps to identify interpretable models by elucidating clusters of objects, variables and hidden layer neurons. Graphical Abstract
Part 2: Concept development of ‘Weather energy transformers (WETRAN)

One of the first thing you likely do in the morning is to look out the window and see what the weather is like. We are interested in harnessing or combatting the weather to capture all sources of energy it produces. The energy, a weather, may encounter from include heat, light, wind, wave and storms. These translational energies are transformed to electrical energy to lighten the homes and buildings; and further to produce green hydrogen to fuel the essential needs of the daily life. Thus, a concept of Weather Energy Transformers (WETRAN) is developed to describe the Consolidated weather energy conversion (CWEC) &amp; Weather forecast energy prediction (WFEP). The WETRAN’s are known as Weather Hybrid Energy Transformers are made primarily to quest the global free energies. WETRAN installations are predicted to come into existence by 2030.
Implementation of the Consciousness-Raising (C-R) Method in Improving EFL Students’ Writing Skills

The purpose of this study was to determine the extent to which the Consciousness-Raising (C-R) method improves students' writing skills and to identify the factors that influence the improvement process. The study was conducted with third-semester students in the English Department, consisting of 16 students. The research design was Classroom Action Research (CAR) and was carried out in three cycles over a period of 12 weeks, with four meetings including one test in each cycle. The cycles comprised of planning, action, observation, and reflection. Qualitative data was obtained through observation, field notes, and interviews, while quantitative data was obtained from students' writing tests. The results of this study showed improvement in the students' writing skills, as demonstrated by the teaching and learning activities throughout each cycle and the increasing scores of the students' writing tests, which went from 65.45 in cycle 1 to 77.63 in cycle 2 and 88.22 in cycle 3. The decrease in errors in each test cycle also supported this improvement. The observation and field notes indicated that the C-R method increased students' attention and awareness, resulting in more focus and engagement in the learning process. The factors that influenced the improvement of students' writing skills were motivation, the role of the instructor, classroom environment, materials, and media. The C-R method was found to be effective in improving students' writing skills and resolving writing problems. The researcher therefore recommends the use of this method, taking into consideration the supporting factors.
Mechanism for Cloud Computing Enviroment's Reliable Au-thenticated Token Handling

Cloud computing (CC) has matured in terms of dependability and effectiveness, which has led to the migration of many applications to the cloud. Three-factor Multilateral Authenticity and Knowledge Acceptance (MAKA) mechanisms for multi-server systems are gaining a lot of interest for their convenience and security. Although there are many extant three-factor MAKA protocols, none of them give a rigorous secure guarantee, making them vulnerable to numerous assaults on other procedures in the chain. And most three-factor MAKA mechanisms lack adaptive cancellation mechanisms, preventing malevolent users from being quickly removed. We present a three-factor MAKA protocol with a proven adaptive reversible evidence in the arbitrary source to overcome these issues. This method uses Schnorr identities to enable vibrant administration by the client. In multi-server setups, our procedure is capable of meeting a wide range of requirements. For intelligent systems with low processing power, the suggested method is a viable option. The entire computation code demonstrates the system's viability.
Unsupervised part-of-speech induction for language description: Modeling documentation materials in Kolyma Yukaghir

This study investigates the clustering of words into Part-of-Speech (POS) classes in Kolyma Yukaghir. In grammatical descriptions, lexical items are assigned to POS classes based on their morphological paradigms. Discursively, however, these classes share a fair amount of morphology. In this study, we turn to POS induction to evaluate if classes based on quantification of the distributions in which roots and affixes are used can be useful for language description purposes, and, if so, what those classes might be. We qualitatively compare clusters of roots and affixes based on four different definitions of their distributions. The results show that clustering is more reliable for words that typically bear more morphology. Additionally, the results suggest that the number of POS classes in Kolyma Yukaghir might be smaller than stated in current descriptions. This study thus demonstrates how unsupervised learning methods can provide insights for language description, particularly for highly inflectional languages.
Enhancing Dialogue Generation with Conversational Concept Flows

Human conversations contain natural and reasonable topic shifts, reflected as the concept flows across utterances.Previous researches prove that explicitly modeling concept flows with a large commonsense knowledge graph effectively improves response quality.However, we argue that there exists a gap between the knowledge graph and the conversation.The knowledge graph has limited commonsense knowledge and ignores the characteristics of natural conversations.Thus, many concepts and relations in conversations are not included.To bridge this gap, we propose to enhance dialogue generation with conversational concept flows.Specifically, we extract abundant concepts and relations from natural conversations and build a new conversation-aware knowledge graph.In addition, we design a novel relation-aware graph encoder to capture the concept flows guided by the knowledge graph.Experimental results on the large-scale Reddit conversation dataset indicate that our method performs better than strong baselines, andfurther analysis verifies the effectiveness of each component.All our code and data will be publicly available after acceptance.
F.K. Sologub and Moscow Publishing House: History of Cooperation

Based on unpublished archival materials and pieces from epistolary sources, the article restores a history of F.K. Sologub’s cooperation with a popular publishing enterprise at the beginning of the 20th century — Moscow Publishing House. The article analyses business relationships between Sologub and Moscow Publishing House staff members: G.G. Blumenberg, the owner and the editor of the literary digests “Zemlia” (20 books, issued by Moscow Publishing House from 1908 to 1917); D.M. Rebrik, Blumenberg trading house’s authorized representative; Bunin, the editor of the first three books of “Zemlia”; V.S. Klestov, the representative of Moscow Publishing House in St. Petersburg. The article follows the history of Sologub’s art works publications in the literary digest “Zemlia” (short stories “The Old House” and “The Beastly Life,” a novel “Smoke and Ashes”) as well as separate editions of some books (short story collections titled “Fierce Year” (1916) and “The Blind Butterfly” (1918), a collection of poems “Red Poppy” (1917)). Based on the reviews of literary critics, the article proves thematic dialogues between the texts by Sologub and other authors within the context of “Zemlia” almanacs. Information on censorship revisions and payment considerations (gradual reduction of fees) appears to be definitely interesting from the perspective of the history of literature.