Reviewerly/exHarmony huggingface dataset:
CIF-PT: Bridging Speech and Text Representations for Spoken Language Understanding via Continuous Integrate-and-Fire Pre-Training

Speech or text representation generated by pre-trained models contains modal-specific information that could be combined for benefiting spoken language understanding (SLU) tasks. In this work, we propose a novel pre-training paradigm termed Continuous Integrate-and-Fire Pre-Training (CIF-PT). It relies on a simple but effective frame-to-token alignment: continuous integrate-and-fire (CIF) to bridge the representations between speech and text. It jointly performs speech-to-text training and language model distillation through CIF as the pre-training (PT). Evaluated on SLU benchmark SLURP dataset, CIF-PT outperforms the state-of-the-art model by 1.94% of accuracy and 2.71% of SLU-F1 on the tasks of intent classification and slot filling, respectively. We also observe the cross-modal representation extracted by CIF-PT obtains better performance than other neural interfaces for the tasks of SLU, including the dominant speech representation learned from self-supervised pre-training.
UIRISC at SemEval-2023 Task 10: Explainable Detection of Online Sexism by Ensembling Fine-tuning Language Models

Under the umbrella of anonymous social networks, many women have suffered from abuse, discrimination, and other sexist expressions online.However, exsiting methods based on keyword filtering and matching performed poorly on online sexism detection, which lacked the capability to identify implicit stereotypes and discrimination.Therefore, this paper proposes a System of Ensembling Fine-tuning Models (SEFM) at SemEval-2023 Task 10: Explainable Detection of Online Sexism.We firstly use four task-adaptive pre-trained language models to flag all texts.Secondly, we alleviate the data imbalance from two perspectives: oversampling the labelled data and adjusting the loss function.Thirdly, we add indicators and feedback modules to enhance the overall performance.Our system attained macro F1 scores of 0.8538, 0.6619, and 0.4641 for Subtask A, B, and C, respectively.Our system exhibited strong performance across multiple tasks, with particularly noteworthy performance in Subtask B. Comparison experiments and ablation studies demonstrate the effectiveness of our system.
Length-Aware NMT and Adaptive Duration for Automatic Dubbing

This paper presents the submission of Huawei Translation Services Center for the IWSLT 2023 dubbing task in the unconstrained setting.The proposed solution consists of a Transformer-based machine translation model and a phoneme duration predictor.The Transformer is deep and multiple target-to-source length-ratio class labels are used to control target lengths.The variation predictor in Fast-Speech2 is utilized to predict phoneme durations.To optimize the isochrony in dubbing, reranking and scaling are performed.The source audio duration is used as a reference to re-rank the translations of different length-ratio labels, and the one with minimum time deviation is preferred.Additionally, the phoneme duration outputs are scaled within a defined threshold to narrow the duration gap with the source audio.
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction

The recent advent of self-supervised pretraining techniques has led to a surge in the use of multimodal learning in form document understanding.However, existing approaches that extend the mask language modeling to other modalities require careful multitask tuning, complex reconstruction target designs, or additional pre-training data.In Form-NetV2, we introduce a centralized multimodal graph contrastive learning strategy to unify self-supervised pre-training for all modalities in one loss.The graph contrastive objective maximizes the agreement of multimodal representations, providing a natural interplay for all modalities without special customization.In addition, we extract image features within the bounding box that joins a pair of tokens connected by a graph edge, capturing more targeted visual cues without loading a sophisticated and separately pre-trained image embedder.FormNetV2 establishes new state-of-theart performance on FUNSD, CORD, SROIE and Payment benchmarks with a more compact model size.
Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models

Grammatical Error Correction (GEC) aims to correct grammatical errors in sentences.We find that autoregressive models tend to assign low probabilities to tokens that need corrections.Here we introduce additional signals to the training of GEC models so that these systems can learn to better predict at ambiguous positions.To do this, we use a non-autoregressive model as an auxiliary model, and develop a new regularization term of training by considering the difference in predictions between the autoregressive and nonautoregressive models.We experiment with this method on both English and Chinese GEC tasks.Experimental results show that our GEC system outperforms the baselines on all the data sets significantly.
Francis Bacon at SemEval-2023 Task 4: Ensembling BERT and GloVe for Value Identification in Arguments

In this paper, we discuss our efforts on SemEval-2023 Task 4, a task to classify the human value categories that an argument draws on.Arguments consist of a premise, conclusion, and the premise's stance on the conclusion.Our team experimented with GloVe embeddings and fine-tuning BERT.We found that an ensembling of BERT and GloVe with RidgeRegression worked the best.
Augmenting Large Language Model Translators via Translation Memories

Using translation memories (TMs) as prompts is a promising approach to in-context learning of machine translation models.In this work, we take a step towards prompting large language models (LLMs) with TMs and making them better translators.We find that the ability of LLMs to "understand" prompts is indeed helpful for making better use of TMs.Experiments show that the results of a pre-trained LLM translator can be greatly improved by using highquality TM-based prompts.These results are even comparable to those of the state-of-the-art NMT systems which have access to large-scale in-domain bilingual data and are well tuned on the downstream tasks.
BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval

Dense retrieval has shown promise in the firststage retrieval process when trained on indomain labeled datasets.However, previous studies have found that dense retrieval is hard to generalize to unseen domains due to its weak modeling of domain-invariant and interpretable feature (i.e., matching signal between two texts, which is the essence of information retrieval).In this paper, we propose a novel method to improve the generalization of dense retrieval via capturing matching signal called BERM.Fully fine-grained expression and query-oriented saliency are two properties of the matching signal.Thus, in BERM, a single passage is segmented into multiple units and two unit-level requirements are proposed for representation as the constraint in training to obtain the effective matching signal.One is semantic unit balance and the other is essential matching unit extractability.Unit-level view and balanced semantics make representation express the text in a fine-grained manner.Essential matching unit extractability makes passage representation sensitive to the given query to extract the pure matching information from the passage containing complex context.Experiments on BEIR show that our method can be effectively combined with different dense retrieval training methods (vanilla, hard negatives mining and knowledge distillation) to improve its generalization ability without any additional inference overhead and target domain data.
Exploring Schema Generalizability of Text-to-SQL

Exploring the generalizability of a text-to-SQL parser is essential for a system to automatically adapt the real-world databases.Previous investigation works mostly focus on lexical diversity, including the influence of the synonym and perturbations in both natural language questions and databases.However, the structural variability of database schema (DS), as a widely seen real-world scenario, is yet underexplored.Specifically, confronted with the same input question, the target SQL may be represented in different ways when the DS comes to a different structure.In this work, we provide indepth discussions about the schema generalizability challenge of text-to-SQL tasks.We observe that current datasets are too templated to study schema generalization.To collect suitable test data, we propose a framework to generate novel text-to-SQL data via automatic and synchronous (DS, SQL) pair altering.When evaluating state-of-the-art text-to-SQL models on the synthetic samples, performance is significantly degraded, which demonstrates the limitation of current research regarding schema generalization.
Ambiguous Learning from Retrieval: Towards Zero-shot Semantic Parsing

Current neural semantic parsers mostly take supervised approaches, which require a considerable amount of expensive training data.As a result, minimizing supervision requirements has been one of the key challenges in semantic parsing.In this paper, we propose a Retrieval as Ambiguous Supervision framework, which can effectively collect high-coverage ambiguous supervisions (i.e., the parse candidates of an utterance) via a pre-trained language modelsbased retrieval system.Then, by assuming candidates will contain the correct ones, the zeroshot task can be converted into an ambiguously supervised task.To improve the precision and coverage of such ambiguous supervision, we propose a confidence-driven self-training algorithm, in which a semantic parser is learned and exploited to disambiguate candidates iteratively.Experimental results show that our approach significantly outperforms the state-of-the-art zero-shot semantic parsing methods.
Seen to Unseen: Exploring Compositional Generalization of Multi-Attribute Controllable Dialogue Generation

Existing controllable dialogue generation work focuses on the single-attribute control and lacks generalization capability to out-of-distribution multiple attribute combinations.In this paper, we explore the compositional generalization for multi-attribute controllable dialogue generation where a model can learn from seen attribute values and generalize to unseen combinations.We propose a prompt-based disentangled controllable dialogue generation model, DCG.It learns attribute concept composition by generating attribute-oriented prompt vectors and uses a disentanglement loss to disentangle different attributes for better generalization.Besides, we design a unified reference-free evaluation framework for multiple attributes with different levels of granularities.Experiment results on two benchmarks prove the effectiveness of our method and the evaluation metric.
CLCL: Non-compositional Expression Detection with Contrastive Learning and Curriculum Learning

Non-compositional expressions present a substantial challenge for natural language processing (NLP) systems, necessitating more intricate processing compared to general language tasks, even with large pre-trained language models.Their non-compositional nature and limited availability of data resources further compound the difficulties in accurately learning their representations.This paper addresses both of these challenges.By leveraging contrastive learning techniques to build improved representations it tackles the non-compositionality challenge.Additionally, we propose a dynamic curriculum learning framework specifically designed to take advantage of the scarce available data for modeling non-compositionality.Our framework employs an easy-to-hard learning strategy, progressively optimizing the model's performance by effectively utilizing available training data.Moreover, we integrate contrastive learning into the curriculum learning approach to maximize its benefits.Experimental results demonstrate the gradual improvement in the model's performance on idiom usage recognition and metaphor detection tasks.Our evaluation encompasses six datasets, consistently affirming the effectiveness of the proposed framework.Our models available at https: //github.com/zhjjn/CLCL.git.
On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection

Detecting adversarial samples that are carefully crafted to fool the model is a critical step to socially-secure applications.However, existing adversarial detection methods require access to sufficient training data, which brings noteworthy concerns regarding privacy leakage and generalizability.In this work, we validate that the adversarial sample generated by attack algorithms is strongly related to a specific vector in the high-dimensional inputs.Such vectors, namely UAPs (Universal Adversarial Perturbations), can be calculated without original training data.Based on this discovery, we propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs.Experimental results show that our method achieves competitive detection performance on various text classification tasks, and maintains an equivalent time consumption to normal inference.
Denoising Enhanced Distantly Supervised Ultrafine Entity Typing

Recently, the task of distantly supervised (DS) ultra-fine entity typing has received significant attention.However, DS data is noisy and often suffers from missing or wrong labeling issues resulting in low precision and low recall.This paper proposes a novel ultra-fine entity typing model with denoising capability.Specifically, we build a noise model to estimate the unknown labeling noise distribution over input contexts and noisy type labels.With the noise model, more trustworthy labels can be recovered by subtracting the estimated noise from the input.Furthermore, we propose an entity typing model, which adopts a bi-encoder architecture, is trained on the denoised data.Finally, the noise model and entity typing model are trained iteratively to enhance each other.We conduct extensive experiments on the Ultra-Fine entity typing dataset as well as OntoNotes dataset and demonstrate that our approach significantly outperforms other baseline methods.
Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequences

Previous studies show that intermediate supervision signals benefit various Natural Language Processing tasks.However, it is not clear whether there exist intermediate signals that benefit Neural Machine Translation (NMT).Borrowing techniques from Statistical Machine Translation, we propose intermediate signals which are intermediate sequences from the "source-like" structure to the "target-like" structure.Such intermediate sequences introduce an inductive bias that reflects a domain-agnostic principle of translation, which reduces spurious correlations that are harmful to out-of-domain generalisation.Furthermore, we introduce a full-permutation multi-task learning to alleviate the spurious causal relations from intermediate sequences to the target, which results from exposure bias.The Minimum Bayes Risk decoding algorithm is used to pick the best candidate translation from all permutations to further improve the performance.Experiments show that the introduced intermediate signals can effectively improve the domain robustness of NMT and reduces the amount of hallucinations on out-of-domain translation.Further analysis shows that our methods are especially promising in low-resource scenarios.
Local Byte Fusion for Neural Machine Translation

Subword tokenization schemes are the dominant technique used in current NLP models.However, such schemes can be rigid and tokenizers built on one corpus may not adapt well to other parallel corpora.It has also been observed that in multilingual corpora, subword tokenization schemes oversegment low-resource languages, leading to a drop in translation performance.An alternative to subword tokenizers is byte-based tokenization, i.e., tokenization into byte sequences using the UTF-8 encoding scheme.Byte tokens often represent inputs at a sub-character granularity, i.e., one character can be represented by a span of byte tokens.This results in much longer byte sequences that are hard to interpret without aggregating local information from multiple byte tokens.In this paper, we propose a Local Byte Fusion (LOBEF) method for byte-based machine translation-utilizing byte n-gram and word boundaries-to aggregate local semantic information.Extensive experiments on multilingual translation, zero-shot cross-lingual transfer, and domain adaptation reveal a consistent improvement over vanilla byte-based models.Further analysis also indicates that our byte-based models are parameter-efficient and perform competitive to subword models.
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics

The proliferation of automatic faithfulness metrics for summarization has produced a need for benchmarks to evaluate them.While existing benchmarks measure the correlation with human judgements of faithfulness on modelgenerated summaries, they are insufficient for diagnosing whether metrics are: 1) consistent, i.e., indicate lower faithfulness as errors are introduced into a summary, 2) effective on human-written texts, and 3) sensitive to different error types (as summaries can contain multiple errors).To address these needs, we present a benchmark of unfaithful minimal pairs (BUMP), a dataset of 889 human-written, minimally different summary pairs, where a single error is introduced to a summary from the CNN/DailyMail dataset to produce an unfaithful summary.We find BUMP complements existing benchmarks in a number of ways: 1) the summaries in BUMP are harder to discriminate and less probable under SOTA summarization models, 2) unlike non-pair-based datasets, BUMP can be used to measure the consistency of metrics, and reveals that the most discriminative metrics tend not to be the most consistent, and 3) unlike datasets containing generated summaries with multiple errors, BUMP enables the measurement of metrics' performance on individual error types.
HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification

Hierarchical text classification (HTC) is a challenging subtask of multi-label classification as the labels form a complex hierarchical structure.Existing dual-encoder methods in HTC achieve weak performance gains with huge memory overheads and their structure encoders heavily rely on domain knowledge.Under such observation, we tend to investigate the feasibility of a memory-friendly model with strong generalization capability that could boost the performance of HTC without prior statistics or label semantics.In this paper, we propose Hierarchy-aware Tree Isomorphism Network (HiTIN) to enhance the text representations with only syntactic information of the label hierarchy.Specifically, we convert the label hierarchy into an unweighted tree structure, termed coding tree, with the guidance of structural entropy.Then we design a structure encoder to incorporate hierarchy-aware information in the coding tree into text representations.Besides the text encoder, HiTIN only contains a few multi-layer perceptions and linear transformations, which greatly saves memory.We conduct experiments on three commonly used datasets and the results demonstrate that HiTIN could achieve better test performance and less memory consumption than state-of-the-art (SOTA) methods.
S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering

Answering multi-hop questions over hybrid factual knowledge from the given text and table (TextTableQA) is a challenging task.Existing models mainly adopt a retriever-reader framework, which have several deficiencies, such as noisy labeling in training retriever, insufficient utilization of heterogeneous information over text and table, and deficient ability for different reasoning operations.In this paper, we propose a three-stage TextTableQA framework S 3 HQA, which comprises of retriever, selector, and reasoner.We use a retriever with refinement training to solve the noisy labeling problem.Then, a hybrid selector considers the linked relationships between heterogeneous data to select the most relevant factual knowledge.For the final stage, instead of adapting a reading comprehension module like in previous methods, we employ a generation-based reasoner to obtain answers.This includes two approaches: a row-wise generator and an LLM prompting generator (first time used in this task).The experimental results demonstrate that our method achieves competitive results in the few-shot setting.When trained on the full dataset, our approach outperforms all baseline methods, ranking first on the HybridQA leaderboard.
NatCS: Eliciting Natural Customer Support Dialogues

Despite growing interest in applications based on natural customer support conversations, there exist remarkably few publicly available datasets that reflect the expected characteristics of conversations in these settings.Existing task-oriented dialogue datasets, which were collected to benchmark dialogue systems mainly in written human-to-bot settings, are not representative of real customer support conversations and do not provide realistic benchmarks for systems that are applied to natural data.To address this gap, we introduce NATCS, a multi-domain collection of spoken customer service conversations.We describe our process for collecting synthetic conversations between customers and agents based on natural language phenomena observed in real conversations.Compared to previous dialogue datasets, the conversations collected with our approach are more representative of real human-to-human conversations along multiple metrics.Finally, we demonstrate potential uses of NATCS, including dialogue act classification and intent induction from conversations as potential applications, showing that dialogue act annotations in NATCS provide more effective training data for modeling real conversations compared to existing synthetic written datasets.We publicly release NATCS to facilitate research in natural dialog systems 1 .
A Semi-Autoregressive Graph Generative Model for Dependency Graph Parsing

Recent years have witnessed the impressive progress in Neural Dependency Parsing.According to the different factorization approaches to the graph joint probabilities, existing parsers can be roughly divided into autoregressive and non-autoregressive patterns.The former means that the graph should be factorized into multiple sequentially dependent components, then it can be built up component by component.And the latter assumes these components to be independent so that they can be outputted in a one-shot manner.However, when treating the directed edge as an explicit dependency relationship, we discover that there is a mixture of independent and interdependent components in the dependency graph, signifying that both aforementioned models fail to precisely capture the explicit dependencies among nodes and edges.Based on this property, we design a Semi-Autoregressive Dependency Parser to generate dependency graphs via adding node groups and edge groups autoregressively while pouring out all group elements in parallel.The model gains a trade-off between non-autoregression and autoregression, which respectively suffer from the lack of target inter-dependencies and the uncertainty of graph generation orders.The experiments show the proposed parser outperforms strong baselines on Enhanced Universal Dependencies of multiple languages, especially achieving 4% average promotion at graph-level accuracy.Also, the performances of model variations show the importance of specific parts.