Reviewerly/exHarmony huggingface dataset:
Efficient Semiring-Weighted Earley Parsing

This paper provides a reference description, in the form of a deduction system, of Earley's (1970) context-free parsing algorithm with various speed-ups.Our presentation includes a known worst-case runtime improvement from Earley's O N 3 |G||R| , which is unworkable for the large grammars that arise in natural language processing, to O N 3 |G| , which matches the runtime of CKY on a binarized version of the grammar G.Here N is the length of the sentence, |R| is the number of productions in G, and |G| is the total length of those productions.We also provide a version that achieves runtime of O N 3 |M| with |M| ≤ |G| when the grammar is represented compactly as a single finite-state automaton M (this is partly novel).We carefully treat the generalization to semiring-weighted deduction, preprocessing the grammar like Stolcke (1995) to eliminate deduction cycles, and further generalize Stolcke's method to compute the weights of sentence prefixes.We also provide implementation details for efficient execution, ensuring that on a preprocessed grammar, the semiring-weighted versions of our methods have the same asymptotic runtime and space requirements as the unweighted methods, including sub-cubic runtime on some grammars.https://github.com/rycolab/earleys-algo 7 All methods in this paper can be also applied directly to lattice parsing, in which i, j, k range over states in an acyclic lattice of possible input strings, and 0 and N refer to the unique initial and final states.A lattice edge from j to k labeled with terminal a is then encoded by the axiom [j, k, a]. 8 Assuming that all nonterminals B ∈ N are generating, i.e., ∃x ′ ∈ Σ * such that B * ⇒ x ′ .To ensure this, repeatedly mark B ∈ N as generating whenever R contains some B → ρ such that all nonterminals in ρ are already marked as generating.Then delete any unmarked nonterminals and their rules. 9Earley (1970) also generalized the algorithm to prove this item only if it can appear in a proof of some string that begins with x 0:(j+∆) , for a fixed ∆.This is lookahead of ∆ tokens.
Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport

Bilingual lexicons form a critical component of various natural language processing applications, including unsupervised and semisupervised machine translation and crosslingual information retrieval.We improve bilingual lexicon induction performance across 40 language pairs with a graph-matching method based on optimal transport.The method is especially strong with low amounts of supervision.
Effective and Efficient Query-aware Snippet Extraction for Web Search

Query-aware webpage snippet extraction is widely used in search engines to help users better understand the content of the returned webpages before clicking.Although important, it is very rarely studied.In this paper, we propose an effective query-aware webpage snippet extraction method named DeepQSE, aiming to select a few sentences which can best summarize the webpage content in the context of input query.DeepQSE first learns query-aware sentence representations for each sentence to capture the fine-grained relevance between query and sentence, and then learns document-aware query-sentence relevance representations for snippet extraction.Since the query and each sentence are jointly modeled in DeepQSE, its online inference may be slow.Thus, we further propose an efficient version of DeepQSE, named Efficient-DeepQSE, which can significantly improve the inference speed of Deep-QSE without affecting its performance.The core idea of Efficient-DeepQSE is to decompose the query-aware snippet extraction task into two stages, i.e., a coarse-grained candidate sentence selection stage where sentence representations can be cached, and a fine-grained relevance modeling stage.Experiments on two real-world datasets validate the effectiveness and efficiency of our methods.
Calibrating Zero-shot Cross-lingual (Un-)structured Predictions

We investigate model calibration in the setting of zero-shot cross-lingual transfer with largescale pre-trained language models.The level of model calibration is an important metric for evaluating the trustworthiness of predictive models.There exists an essential need for model calibration when natural language models are deployed in critical tasks.We study different post-training calibration methods in structured and unstructured prediction tasks.We find that models trained with data from the source language become less calibrated when applied to the target language and that calibration errors increase with intrinsic task difficulty and relative sparsity of training data.Moreover, we observe a potential connection between the level of calibration error and an earlier proposed measure of the distance from English to other languages.Finally, our comparison demonstrates that among other methods Temperature Scaling (TS) generalizes well to distant languages, but TS fails to calibrate more complex confidence estimation in structured predictions compared to more expressive alternatives like Gaussian Process Calibration.
Generative Entity Typing with Curriculum Learning

Entity typing aims to assign types to the entity mentions in given texts.The traditional classification-based entity typing paradigm has two unignorable drawbacks: 1) it fails to assign an entity to the types beyond the predefined type set, and 2) it can hardly handle few-shot and zero-shot situations where many long-tail types only have few or even no training instances.To overcome these drawbacks, we propose a novel generative entity typing (GET) paradigm: given a text with an entity mention, the multiple types for the role that the entity plays in the text are generated with a pre-trained language model (PLM).However, PLMs tend to generate coarse-grained types after finetuning upon the entity typing dataset.In addition, only the heterogeneous training data consisting of a small portion of human-annotated data and a large portion of auto-generated but low-quality data are provided for model training.To tackle these problems, we employ curriculum learning (CL) to train our GET model on heterogeneous data, where the curriculum could be self-adjusted with the self-paced learning according to its comprehension of the type granularity and data heterogeneity.Our extensive experiments upon the datasets of different languages and downstream tasks justify the superiority of our GET model over the state-ofthe-art entity typing models.The code has been released on https://github.com/siyuyuan/GET.
Toward the Limitation of Code-Switching in Cross-Lingual Transfer

Multilingual pretrained models have shown strong cross-lingual transfer ability.Some works used code-switching sentences, which consist of tokens from multiple languages, to enhance the cross-lingual representation further, and have shown success in many zero-shot cross-lingual tasks.However, code-switched tokens are likely to cause grammatical incoherence in newly substituted sentences, and negatively affect the performance on tokensensitive tasks, such as Part-of-Speech (POS) tagging and Named-Entity-Recognition (NER).This paper mitigates the limitation of the codeswitching method by not only making the token replacement but considering the similarity between the context and the switched tokens so that the newly substituted sentences are grammatically consistent during both training and inference.We conduct experiments on crosslingual POS and NER over 30+ languages, and demonstrate the effectiveness of our method by outperforming the mBERT by 0.95 and original code-switching method by 1.67 on F1 scores.How do menschen look at and अनु भव 艺术 ?How do people look at and experience art ?How do people look at and experience art ?How do Menschen look at and अनु भव 艺术 ?
Analyzing the Representational Geometry of Acoustic Word Embeddings

Acoustic word embeddings (AWEs) are vector representations such that different acoustic exemplars of the same word are projected nearby in the embedding space.In addition to their use in speech technology applications such as spoken term discovery and keyword spotting, AWE models have been adopted as models of spoken-word processing in several cognitively motivated studies and have been shown to exhibit human-like performance in some auditory processing tasks.Nevertheless, the representational geometry of AWEs remains an underexplored topic that has not been studied in the literature.In this paper, we take a closer analytical look at AWEs learned from English speech and study how the choice of the learning objective and the architecture shapes their representational profile.To this end, we employ a set of analytic techniques from machine learning and neuroscience in three different analyses: embedding space uniformity, word discriminability, and representational consistency.Our main findings highlight the prominent role of the learning objective on shaping the representation profile compared to the model architecture.
Distill The Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation

Past works on multimodal machine translation (MMT) elevate bilingual setup by incorporating additional aligned vision information.However, an image-must requirement of the multimodal dataset largely hinders MMT's development -namely that it demands an aligned form of [image, source text, target text].This limitation is generally troublesome during the inference phase especially when the aligned image is not provided as in the normal NMT setup.Thus, in this work, we introduce IKD-MMT, a novel MMT framework to support the image-free inference phase via an inversion knowledge distillation scheme.In particular, a multimodal feature generator is executed with a knowledge distillation module, which directly generates the multimodal feature from (only) source texts as the input.While there have been a few prior works entertaining the possibility to support image-free inference for machine translation, their performances have yet to rival the image-must translation.In our experiments, we identify our method as the first image-free approach to comprehensively rival or even surpass (almost) all image-must frameworks, and achieved the state-of-the-art result on the often-used Multi30k benchmark 1 .
A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion

Recent work has demonstrated that entity representations can be extracted from pre-trained language models to develop knowledge graph completion models that are more robust to the naturally occurring sparsity found in knowledge graphs.In this work, we conduct a comprehensive exploration of how to best extract and incorporate those embeddings into knowledge graph completion models.We explore the suitability of the extracted embeddings for direct use in entity ranking and introduce both unsupervised and supervised processing methods that can lead to improved downstream performance.We then introduce supervised embedding extraction methods that can extract more informative representations.We then synthesize our findings and develop a knowledge graph completion model that significantly outperforms recent neural models.
Scientific Paper Extractive Summarization Enhanced by Citation Graphs

In a citation graph, adjacent paper nodes share related scientific terms and topics.The graph thus conveys unique structure information of document-level relatedness that can be utilized in the paper summarization task, for exploring beyond the intra-document information.In this work, we focus on leveraging citation graphs to improve scientific paper extractive summarization under different settings.We first propose a Multi-granularity Unsupervised Summarization model (MUS) as a simple and low-cost solution to the task.MUS finetunes a pre-trained encoder model on the citation graph by link prediction tasks.Then, the abstract sentences are extracted from the corresponding paper considering multi-granularity information.Preliminary results demonstrate that citation graph is helpful even in a simple unsupervised framework.Motivated by this, we next propose a Graph-based Supervised Summarization model (GSS) to achieve more accurate results on the task when large-scale labeled data are available.Apart from employing the link prediction as an auxiliary task, GSS introduces a gated sentence encoder and a graph information fusion module to take advantage of the graph information to polish the sentence representation.Experiments on a public benchmark dataset show that MUS and GSS bring substantial improvements over the prior state-of-the-art model.
Mixture of Attention Heads: Selecting Attention Heads Per Token

Mixture-of-Experts (MoE) networks have been proposed as an efficient way to scale up model capacity and implement conditional computing.However, the study of MoE components mostly focused on the feedforward layer in Transformer architecture.This paper proposes the Mixture of Attention Heads (MoA), a new architecture that combines multi-head attention with the MoE mechanism.MoA includes a set of attention heads that each has its own set of parameters.Given an input, a router dynamically selects a subset of k attention heads per token.This conditional computation schema allows MoA to achieve stronger performance than the standard multi-head attention layer.Furthermore, the sparsely gated MoA can easily scale up the number of attention heads and the number of parameters while preserving computational efficiency.In addition to the performance improvements, MoA also automatically differentiates heads' utilities, providing a new perspective to discuss the model's interpretability.We conducted experiments on several important tasks, including Machine Translation and Masked Language Modeling.Experiments have shown promising results on several tasks against strong baselines that involve large and very deep models 1 .
ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic-English

We present our work on collecting ArzEn-ST, a code-switched Egyptian Arabic -English Speech Translation Corpus.This corpus is an extension of the ArzEn speech corpus, which was collected through informal interviews with bilingual speakers.In this work, we collect translations in both directions, monolingual Egyptian Arabic and monolingual English, forming a three-way speech translation corpus.We make the translation guidelines and corpus publicly available.We also report results for baseline systems for machine translation and speech translation tasks.We believe this is a valuable resource that can motivate and facilitate further research studying the codeswitching phenomenon from a linguistic perspective and can be used to train and evaluate NLP systems.
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup

Training a model to provide natural language explanations (NLEs) for its predictions usually requires the acquisition of task-specific NLEs, which is time-and resource-consuming.A potential solution is the few-shot out-of-domain transfer of NLEs from a parent task with many NLEs to a child task.In this work, we examine the setup in which the child task has few NLEs but abundant labels.We establish four few-shot transfer learning methods that cover the possible fine-tuning combinations of the labels and NLEs for the parent and child tasks.We transfer explainability from a large natural language inference dataset (e-SNLI) separately to two child tasks: (1) hard cases of pronoun resolution, where we introduce the small-e-WinoGrande dataset of NLEs on top of the WinoGrande dataset, and (2) commonsense validation (ComVE).Our results demonstrate that the parent task helps with NLE generation and we establish the best methods for this setup.
Towards relation extraction from speech

Relation extraction has focused on extracting semantic relationships between entities from the unstructured written textual data.However, with the vast and rapidly increasing amounts of spoken data, relation extraction from speech is an important but under-explored problem.In this paper, we propose a new information extraction task, speech relation extraction (SpeechRE).To facilitate further research, we construct the first synthetic training datasets, as well as the first human-spoken test set with native English speakers.We establish strong baseline performance for SpeechRE via two approaches.The pipeline approach connects a pretrained ASR module with a text-based relation extraction module.The end-to-end approach employs a cross-modal encoder-decoder architecture.Our comprehensive experiments reveal the relative strengths and weaknesses of these approaches, and shed light on important future directions in SpeechRE research.We share the source code and datasets on https://github.com/wutong8023/SpeechRE.
RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval

Video language pre-training methods have mainly adopted sparse sampling techniques to alleviate the temporal redundancy of videos.Though effective, sparse sampling still suffers inter-modal redundancy: visual redundancy and textual redundancy.Compared with highly generalized text, sparsely sampled frames usually contain text-independent portions, called visual redundancy.Sparse sampling is also likely to miss important frames corresponding to some text portions, resulting in textual redundancy.Inter-modal redundancy leads to a mismatch of video and text information, hindering the model from better learning the shared semantics across modalities.To alleviate it, we propose Redundancy-aware Video-language Pre-training.We design a redundancy measurement of video patches and text tokens by calculating the cross-modal minimum dis-similarity.Then, we penalize the high-redundant video patches and text tokens through a proposed redundancy-aware contrastive learning.We evaluate our method on four benchmark datasets, MSRVTT, MSVD, DiDeMo, and LSMDC, achieving a significant improvement over the previous state-ofthe-art results.
Questioning the Validity of Summarization Datasets and Improving Their Factual Consistency

The topic of summarization evaluation has recently attracted a surge of attention due to the rapid development of abstractive summarization systems.However, the formulation of the task is rather ambiguous, neither the linguistic nor the natural language processing community has succeeded in giving a mutually agreed-upon definition.Due to this lack of well-defined formulation, a large number of popular abstractive summarization datasets are constructed in a manner that neither guarantees validity nor meets one of the most essential criteria of summarization: factual consistency.In this paper, we address this issue by combining state-of-the-art factual consistency models to identify the problematic instances present in popular summarization datasets.We release SummFC, a filtered summarization dataset with improved factual consistency, and demonstrate that models trained on this dataset achieve improved performance in nearly all quality aspects.We argue that our dataset should become a valid benchmark for developing and evaluating summarization systems.
Multimodal Robustness for Neural Machine Translation

In this paper, we look at the case of a Generic text-to-text NMT model that has to deal with data coming from various modalities, like speech, images, or noisy text extracted from the web.We propose a two-step method, based on composable adapters, to deal with this problem of Multimodal Robustness.In the first step, we separately learn domain adapters and modality specific adapters, to deal with noisy input coming from various sources: ASR, OCR, or noisy text (UGC).In a second step, we combine these components at runtime via dynamic routing or, when the source of noise is unknown, via two new transfer learning mechanisms (Fast Fusion and Multi Fusion).We show that our method provides a flexible, state-of-the-art, architecture able to deal with noisy multimodal inputs.
Residual Learning of Neural Text Generation with n-gram Language Model

N -gram language models (LM) have been largely superseded by neural LMs as the latter exhibits better performance.However, we find that n-gram models can achieve satisfactory performance on a large proportion of testing cases, indicating they have already captured abundant knowledge of the language with relatively low computational cost.With this observation, we propose to learn a neural LM that fits the residual between an n-gram LM and the real-data distribution.The combination of n-gram and neural LMs not only allows the neural part to focus on the deeper understanding of language but also provides a flexible way to customize an LM by switching the underlying ngram model without changing the neural model.Experimental results on three typical language tasks (i.e., language modeling, machine translation, and summarization) demonstrate that our approach attains additional performance gains over popular standalone neural models consistently.We also show that our approach allows for effective domain adaptation by simply switching to a domain-specific n-gram model, without any extra training.Our code is released at https://github.com/ghrua/NgramRes.
CQR-SQL: Conversational Question Reformulation Enhanced Context-Dependent Text-to-SQL Parsers

Context-dependent text-to-SQL is the task of translating multi-turn questions into databaserelated SQL queries.Existing methods typically focus on making full use of history context or previously predicted SQL for currently SQL parsing, while neglecting to explicitly comprehend the schema and conversational dependency, such as co-reference, ellipsis and user focus change.In this paper, we propose CQR-SQL, which uses auxiliary Conversational Question Reformulation (CQR) learning to explicitly exploit schema and decouple contextual dependency for multi-turn SQL parsing.Specifically, we first present a schema enhanced recursive CQR method to produce domain-relevant self-contained questions.Secondly, we train CQR-SQL models to map the semantics of multi-turn questions and auxiliary self-contained questions into the same latent space through schema grounding consistency task and tree-structured SQL parsing consistency task, which enhances the abilities of SQL parsing by adequately contextual understanding.At the time of writing, our CQR-SQL achieves new state-of-the-art results on two context-dependent text-to-SQL benchmarks SPARC and COSQL.
VisToT: Vision-Augmented Table-to-Text Generation

Table-to-text generation has been widely studied in the Natural Language Processing community in the recent years.We give a new perspective to this problem by incorporating signals from both tables as well as associated images to generate relevant text.While tables contain a structured list of facts, images are a rich source of unstructured visual information.For example, in the tourism domain, images can be used to infer knowledge such as the type of landmark (e.g., church), its architecture (e.g., Ancient Roman), and composition (e.g., white marble).Therefore, in this paper, we introduce the novel task of Vision-augmented Table-To-Text Generation (VISTOT), defined as follows: given a table and an associated image, produce a descriptive sentence conditioned on the multimodal input.For the task, we present a novel multimodal table-to-text dataset, WIKILAND-MARKS, covering 73,084 unique world landmarks.Further, we also present a competitive architecture, namely, VT3 that generates accurate sentences conditioned on the image and table pairs.Through extensive analyses and experiments, we show that visual cues from images are helpful in (i) inferring missing information from incomplete or sparse tables, and (ii) strengthening the importance of useful information from noisy tables for natural language generation.We make the code and data publicly available 1 .1