pwc-archive/papers-with-abstracts huggingface dataset:
Intrinsic Self-correction for Enhanced Morality: An Analysis of Internal Mechanisms and the Superficial Hypothesis
Authors: Guangliang Liu, Haitao Mao, Jiliang Tang, Kristen Marie Johnson
Date: 2024-07-21 00:00:00
Tasks: Question Answering, Text Generation

Large Language Models (LLMs) are capable of producing content that perpetuates stereotypes, discrimination, and toxicity. The recently proposed moral self-correction is a computationally efficient method for reducing harmful content in the responses of LLMs. However, the process of how injecting self-correction instructions can modify the behavior of LLMs remains under-explored. In this paper, we explore the effectiveness of moral self-correction by answering three research questions: (1) In what scenarios does moral self-correction work? (2) What are the internal mechanisms of LLMs, e.g., hidden states, that are influenced by moral self-correction instructions? (3) Is intrinsic moral self-correction actually superficial in terms of reduced immorality in hidden states? We argue that self-correction can help LLMs find a shortcut to more morally correct output, rather than truly reducing the immorality stored in hidden states. Through empirical investigation with tasks of language generation and multi-choice question answering, we conclude:(i) LLMs exhibit good performance across both tasks, and self-correction instructions are particularly beneficial when the correct answer is already top-ranked; (ii) The morality levels in intermediate hidden states are strong indicators as to whether one instruction would be more effective than another; (iii) Based on our analysis of intermediate hidden states and task case studies of self-correction behaviors, we are first to propose the hypothesis that intrinsic moral self-correction is in fact superficial.
Super-efficiency and Stock Market Valuation: Evidence from Listed Banks in China (2006 to 2023)
Authors: Yun Liao
Date: 2024-07-20 00:00:00

This study investigates the relationship between bank efficiency and stock market valuation using an unbalanced panel dataset of 42 listed banks in China from 2006 to 2023. We employ a non-radial and non-oriented slack based super-efficiency Data Envelopment Analysis (Super-SBM-UND-VRS based DEA) model, which treats Non-Performing Loans (NPLs) as an undesired output. Our results show that the relationship between super-efficiency and stock market valuation is stronger than that between Return on Asset (ROA) and stock market performance, as measured by Tobin's Q. Notably, the Super-SBM-UND-VRS model yields novel results compared to other efficiency methods, such as the Stochastic Frontier Analysis (SFA) approach and traditional DEA models. Furthermore, our results suggest that bank evaluations benefit from decreased ownership concentration, whereas interest rate liberalization has the opposite effect.
SynCPKL: Harnessing LLMs to Generate Synthetic Data for Commonsense Persona Knowledge Linking
Authors: Kuan-Yen Lin
Date: 2024-07-21 00:00:00

Understanding rich dialogues often requires NLP systems to access relevant commonsense persona knowledge, but retrieving this knowledge is challenging due to complex contexts and the implicit nature of commonsense. This paper presents our approach to the Commonsense Persona Knowledge Linking (CPKL) challenge, addressing the critical need for integrating persona and commonsense knowledge in open-domain dialogue systems. We introduce SynCPKL Pipeline, a pipeline that leverages Large Language Models to generate high-quality synthetic datasets for training commonsense persona knowledge linkers. To demonstrate the efficacy of our approach, we present SynCPKL, a new dataset specifically designed for this task. Our experiments validate the effectiveness of SynCPKL for training commonsense persona knowledge linkers. Additionally, our top-performing model, Derberta-SynCPKL, secured first place in the CPKL challenge by a 16% improvement in F1 score. We released both SynCPKL and Derberta-SynCPKL at https://github.com/irislin1006/CPKL.
Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation
Authors: Liwen Sun, James Zhao, Megan Han, Chenyan Xiong
Date: 2024-07-21 00:00:00
Tasks: Diagnostic, RAG, Retrieval, Text Generation

Multimodal foundation models hold significant potential for automating radiology report generation, thereby assisting clinicians in diagnosing cardiac diseases. However, generated reports often suffer from serious factual inaccuracy. In this paper, we introduce a fact-aware multimodal retrieval-augmented pipeline in generating accurate radiology reports (FactMM-RAG). We first leverage RadGraph to mine factual report pairs, then integrate factual knowledge to train a universal multimodal retriever. Given a radiology image, our retriever can identify high-quality reference reports to augment multimodal foundation models, thus enhancing the factual completeness and correctness of report generation. Experiments on two benchmark datasets show that our multimodal retriever outperforms state-of-the-art retrievers on both language generation and radiology-specific metrics, up to 6.5% and 2% score in F1CheXbert and F1RadGraph. Further analysis indicates that employing our factually-informed training strategy imposes an effective supervision signal, without relying on explicit diagnostic label guidance, and successfully propagates fact-aware capabilities from the multimodal retriever to the multimodal foundation model in radiology report generation.
XAI meets LLMs: A Survey of the Relation between Explainable AI and Large Language Models
Authors: Erik Cambria, Lorenzo Malandri, Fabio Mercorio, Navid Nobani, Andrea Seveso
Date: 2024-07-21 00:00:00
Tasks: Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Relation, Survey

In this survey, we address the key challenges in Large Language Models (LLM) research, focusing on the importance of interpretability. Driven by increasing interest from AI and business sectors, we highlight the need for transparency in LLMs. We examine the dual paths in current LLM research and eXplainable Artificial Intelligence (XAI): enhancing performance through XAI and the emerging focus on model interpretability. Our paper advocates for a balanced approach that values interpretability equally with functional advancements. Recognizing the rapid development in LLM research, our survey includes both peer-reviewed and preprint (arXiv) papers, offering a comprehensive overview of XAI's role in LLM research. We conclude by urging the research community to advance both LLM and XAI fields together.
Two eyes, Two views, and finally, One summary! Towards Multi-modal Multi-tasking Knowledge-Infused Medical Dialogue Summarization
Authors: Anisha Saha, Abhisek Tiwari, Sai Ruthvik, Sriparna Saha
Date: 2024-07-21 00:00:00
Tasks: Chunking, Conversation Summarization, dialogue summary

We often summarize a multi-party conversation in two stages: chunking with homogeneous units and summarizing the chunks. Thus, we hypothesize that there exists a correlation between homogeneous speaker chunking and overall summarization tasks. In this work, we investigate the effectiveness of a multi-faceted approach that simultaneously produces summaries of medical concerns, doctor impressions, and an overall view. We introduce a multi-modal, multi-tasking, knowledge-infused medical dialogue summary generation (MMK-Summation) model, which is incorporated with adapter-based fine-tuning through a gated mechanism for multi-modal information integration. The model, MMK-Summation, takes dialogues as input, extracts pertinent external knowledge based on the context, integrates the knowledge and visual cues from the dialogues into the textual content, and ultimately generates concise summaries encompassing medical concerns, doctor impressions, and a comprehensive overview. The introduced model surpasses multiple baselines and traditional summarization models across all evaluation metrics (including human evaluation), which firmly demonstrates the efficacy of the knowledge-guided multi-tasking, multimodal medical conversation summarization. The code is available at https://github.com/NLP-RL/MMK-Summation.
A Survey on Employing Large Language Models for Text-to-SQL Tasks
Authors: Liang Shi, Zhengju Tang, Nan Zhang, Xiaotong Zhang, Zhi Yang
Date: 2024-07-21 00:00:00
Tasks: Natural Language Queries, Prompt Engineering, SQL Parsing, Text to SQL, Text-To-SQL

The increasing volume of data in relational databases and the expertise needed for writing SQL queries pose challenges for users to access and analyze data. Text-to-SQL (Text2SQL) solves the issues by utilizing natural language processing (NLP) techniques to convert natural language into SQL queries. With the development of Large Language Models (LLMs), a range of LLM-based Text2SQL methods have emerged. This survey provides a comprehensive review of LLMs in Text2SQL tasks. We review benchmark datasets, prompt engineering methods, fine-tuning methods, and base models in LLM-based Text2SQL methods. We provide insights in each part and discuss future directions in this field.
Analyzing selected cryptocurrencies spillover effects on global financial indices: Comparing risk measures using conventional and eGARCH-EVT-Copula approaches
Authors: Shafique Ur Rehman, Touqeer Ahmad, Wu Dash Desheng, Amirhossein Karamoozian
Date: 2024-07-22 00:00:00

This study examines the interdependence between cryptocurrencies and international financial indices, such as MSCI World and MSCI Emerging Markets. We compute the value at risk, expected shortfall (ES), and range value at risk (RVaR) and investigate the dynamics of risk spillover. We employ a hybrid approach to derive these risk measures that integrate GARCH models, extreme value models, and copula functions. This framework uses a bivariate portfolio approach involving cryptocurrency data and traditional financial indices. To estimate the above risks of these portfolio structures, we employ symmetric and asymmetric GARCH and both tail flexible EVT models as marginal to model the marginal distribution of each return series and apply different copula functions to connect the pairs of marginal distributions into a multivariate distribution. The empirical findings indicate that the eGARCH EVT-based copula model adeptly captures intricate dependencies, surpassing conventional methodologies like Historical simulations and t-distributed parametric in VaR estimation. At the same time, the HS method proves superior for ES, and the t-distributed parametric method outperforms RVaR. Eventually, the Diebold-Yilmaz approach will be applied to compute risk spillovers between four sets of asset sequences. This phenomenon implies that cryptocurrencies reveal substantial spillover effects among themselves but minimal impact on other assets. From this, it can be concluded that cryptocurrencies propose diversification benefits and do not provide hedging advantages within an investor's portfolio. Our results underline RVaR superiority over ES regarding regulatory arbitrage and model misspecification. The conclusions of this study will benefit investors and financial market professionals who aspire to comprehend digital currencies as a novel asset class and attain perspicuity in regulatory arbitrage.
Fine-grained Gender Control in Machine Translation with Large Language Models
Authors: Minwoo Lee, Hyukhun Koh, Minsung Kim, Kyomin Jung
Date: 2024-07-21 00:00:00
Tasks: Machine Translation, Sentence, Translation

In machine translation, the problem of ambiguously gendered input has been pointed out, where the gender of an entity is not available in the source sentence. To address this ambiguity issue, the task of controlled translation that takes the gender of the ambiguous entity as additional input have been proposed. However, most existing works have only considered a simplified setup of one target gender for input. In this paper, we tackle controlled translation in a more realistic setting of inputs with multiple entities and propose Gender-of-Entity (GoE) prompting method for LLMs. Our proposed method instructs the model with fine-grained entity-level gender information to translate with correct gender inflections. By utilizing four evaluation benchmarks, we investigate the controlled translation capability of LLMs in multiple dimensions and find that LLMs reach state-of-the-art performance in controlled translation. Furthermore, we discover an emergence of gender interference phenomenon when controlling the gender of multiple entities. Finally, we address the limitations of existing gender accuracy evaluation metrics and propose leveraging LLMs as an evaluator for gender inflection in machine translation.
A multi-level multi-label text classification dataset of 19th century Ottoman and Russian literary and critical texts
Authors: Gokcen Gokceoglu, Devrim Cavusoglu, Emre Akbas, Özen Nergis Dolcerocca
Date: 2024-07-21 00:00:00
Tasks: Articles, Multi Label Text Classification, Multi-Label Text Classification, text-classification, Text Classification

This paper introduces a multi-level, multi-label text classification dataset comprising over 3000 documents. The dataset features literary and critical texts from 19th-century Ottoman Turkish and Russian. It is the first study to apply large language models (LLMs) to this dataset, sourced from prominent literary periodicals of the era. The texts have been meticulously organized and labeled. This was done according to a taxonomic framework that takes into account both their structural and semantic attributes. Articles are categorized and tagged with bibliometric metadata by human experts. We present baseline classification results using a classical bag-of-words (BoW) naive Bayes model and three modern LLMs: multilingual BERT, Falcon, and Llama-v2. We found that in certain cases, Bag of Words (BoW) outperforms Large Language Models (LLMs), emphasizing the need for additional research, especially in low-resource language settings. This dataset is expected to be a valuable resource for researchers in natural language processing and machine learning, especially for historical and low-resource languages. The dataset is publicly available^1.
Training Zero-Shot Generalizable End-to-End Task-Oriented Dialog System Without Turn-level Dialog Annotations
Authors: Adib Mosharrof, A. B. Siddique
Date: 2024-07-21 00:00:00
Tasks: Information Retrieval, slot-filling, Slot Filling

Task-oriented dialogue (TOD) systems enable users to achieve their goals through natural language interactions. Traditionally, these systems have relied on turn-level manually annotated metadata, such as dialogue states and policy annotations, which are expensive, time-consuming, and often inconsistent or error-prone. This dependence limits the potential to leverage vast amounts of readily available conversational data for training TOD systems. Additionally, a critical challenge in TOD system design is determining when and how to access and integrate information from external sources. Current approaches typically expect this information to be provided alongside the dialogue context, rather than learning to identify and retrieve it autonomously. While pre-trained large language models (LLMs) have been used to develop TOD systems, their potential to train such systems without laborious annotations remains largely unexplored. This work employs multi-task instruction fine-tuning to create more efficient and scalable TOD systems that can effectively leverage natural language conversational data without manual annotations, while autonomously managing external information retrieval. Our extensive experimental evaluations, using three diverse TOD datasets and three LLMs of varying sizes, demonstrate that our approach can generalize to new, unseen domains. Notably, our approach outperforms both state-of-the-art models trained on annotated data and billion-scale parameter off-the-shelf ChatGPT models.
Enhancing Incremental Summarization with Structured Representations
Authors: EunJeong Hwang, Yichao Zhou, James Bradley Wendt, Beliz Gunel, Nguyen Vo, Jing Xie, Sandeep Tata
Date: 2024-07-21 00:00:00

Large language models (LLMs) often struggle with processing extensive input contexts, which can lead to redundant, inaccurate, or incoherent summaries. Recent methods have used unstructured memory to incrementally process these contexts, but they still suffer from information overload due to the volume of unstructured data handled. In our study, we introduce structured knowledge representations ($GU_{json}$), which significantly improve summarization performance by 40% and 14% across two public datasets. Most notably, we propose the Chain-of-Key strategy ($CoK_{json}$) that dynamically updates or augments these representations with new information, rather than recreating the structured memory for each new source. This method further enhances performance by 7% and 4% on the datasets.
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
Authors: Yongxin Huang, Kexin Wang, Goran Glavaš, Iryna Gurevych
Date: 2024-07-20 00:00:00
Tasks: Contrastive Learning, Multiple-choice, Sentence, Sentence Embeddings, text similarity

Multilingual sentence encoders are commonly obtained by training multilingual language models to map sentences from different languages into a shared semantic space. As such, they are subject to curse of multilinguality, a loss of monolingual representational accuracy due to parameter sharing. Another limitation of multilingual sentence encoders is the trade-off between monolingual and cross-lingual performance. Training for cross-lingual alignment of sentence embeddings distorts the optimal monolingual structure of semantic spaces of individual languages, harming the utility of sentence embeddings in monolingual tasks. In this work, we address both issues by modular training of sentence encoders, i.e., by separating monolingual specialization from cross-lingual alignment. We first efficiently train language-specific sentence encoders to avoid negative interference between languages (i.e., the curse). We then align all non-English monolingual encoders to the English encoder by training a cross-lingual alignment adapter on top of each, preventing interference with monolingual specialization from the first step. In both steps, we resort to contrastive learning on machine-translated paraphrase data. Monolingual and cross-lingual evaluations on semantic text similarity/relatedness and multiple-choice QA render our modular solution more effective than multilingual sentence encoders, especially benefiting low-resource languages.
Seal: Advancing Speech Language Models to be Few-Shot Learners
Authors: Shuyu Lei, Lingen Liu, Jiaolong Yang, Yasen Jiao, Yuxiang Yang, Yushu Yang, Xiang Guo
Date: 2024-07-20 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling

Existing auto-regressive language models have demonstrated a remarkable capability to perform a new task with just a few examples in prompt, without requiring any additional training. In order to extend this capability to a multi-modal setting (i.e. speech and language), this paper introduces the Seal model, an abbreviation for speech language model. It incorporates a novel alignment method, in which Kullback-Leibler divergence loss is performed to train a projector that bridges a frozen speech encoder with a frozen language model decoder. The resulting Seal model exhibits robust performance as a few-shot learner on two speech understanding tasks. Additionally, consistency experiments are conducted to validate its robustness on different pre-trained language models.
Overview of AI-Debater 2023: The Challenges of Argument Generation Tasks
Authors: Jiayu Lin, Guanrong Chen, Bojun Jin, Chenyang Li, Shutong Jia, Wancong Lin, Yang Sun, Yuhang He, Caihua Yang, Jianzhu Bao, Jipeng Wu, Wen Su, Jinglu Chen, Xinyi Li, Tianyu Chen, Mingjie Han, Shuaiwen Du, Zijian Wang, Jiyin Li, Fuzhong Suo, Hao Wang, Nuanchen Lin, Xuanjing Huang, Changjian Jiang, Ruifeng Xu, Long Zhang, Jiuxin Cao, Ting Jin, Zhongyu Wei
Date: 2024-07-20 00:00:00

In this paper we present the results of the AI-Debater 2023 Challenge held by the Chinese Conference on Affect Computing (CCAC 2023), and introduce the related datasets. We organize two tracks to handle the argumentative generation tasks in different scenarios, namely, Counter-Argument Generation (Track 1) and Claim-based Argument Generation (Track 2). Each track is equipped with its distinct dataset and baseline model respectively. In total, 32 competing teams register for the challenge, from which we received 11 successful submissions. In this paper, we will present the results of the challenge and a summary of the systems, highlighting commonalities and innovations among participating systems. Datasets and baseline models of the AI-Debater 2023 Challenge have been already released and can be accessed through the official website of the challenge.
Text Style Transfer: An Introductory Overview
Authors: Sourabrata Mukherjee, Ondrej Dušek
Date: 2024-07-20 00:00:00
Tasks: Style Transfer, Text Generation, Text Style Transfer

Text Style Transfer (TST) is a pivotal task in natural language generation to manipulate text style attributes while preserving style-independent content. The attributes targeted in TST can vary widely, including politeness, authorship, mitigation of offensive language, modification of feelings, and adjustment of text formality. TST has become a widely researched topic with substantial advancements in recent years. This paper provides an introductory overview of TST, addressing its challenges, existing approaches, datasets, evaluation measures, subtasks, and applications. This fundamental overview improves understanding of the background and fundamentals of text style transfer.
Automatic Real-word Error Correction in Persian Text
Authors: Seyed Mohammad Sadegh Dashti, Amid Khatibi Bardsiri, Mehdi Jafari Shahbazzadeh
Date: 2024-07-20 00:00:00
Tasks: feature selection, Semantic Similarity, Semantic Textual Similarity, Spelling Correction

Automatic spelling correction stands as a pivotal challenge within the ambit of natural language processing (NLP), demanding nuanced solutions. Traditional spelling correction techniques are typically only capable of detecting and correcting non-word errors, such as typos and misspellings. However, context-sensitive errors, also known as real-word errors, are more challenging to detect because they are valid words that are used incorrectly in a given context. The Persian language, characterized by its rich morphology and complex syntax, presents formidable challenges to automatic spelling correction systems. Furthermore, the limited availability of Persian language resources makes it difficult to train effective spelling correction models. This paper introduces a cutting-edge approach for precise and efficient real-word error correction in Persian text. Our methodology adopts a structured, multi-tiered approach, employing semantic analysis, feature selection, and advanced classifiers to enhance error detection and correction efficacy. The innovative architecture discovers and stores semantic similarities between words and phrases in Persian text. The classifiers accurately identify real-word errors, while the semantic ranking algorithm determines the most probable corrections for real-word errors, taking into account specific spelling correction and context properties such as context, semantic similarity, and edit-distance measures. Evaluations have demonstrated that our proposed method surpasses previous Persian real-word error correction models. Our method achieves an impressive F-measure of 96.6% in the detection phase and an accuracy of 99.1% in the correction phase. These results clearly indicate that our approach is a highly promising solution for automatic real-word error correction in Persian text.