Reviewerly/exHarmony huggingface dataset:
INFORM : Information eNtropy based multi-step reasoning FOR large language Models

Large language models (LLMs) have demonstrated exceptional performance in reasoning tasks with dedicated Chain-of-Thought (CoT) prompts. Further enhancing CoT prompts with exquisite exemplars can significantly improve reasoning performance.However, the effectiveness of CoT prompts may fluctuate dramatically with different choices of in-context examples. Additionally, manual construction of rationale steps can be time-consuming, presenting challenges for the widespread adoption of CoT prompting. In this work, we propose a novel approach by introducing information entropy (IE) as a criteria on for CoT prompt selection. We extend this criterion to the CoT generation and inference stages, automatically generating CoT prompts with higher information entropy scores and adaptively determining the number of samples. These three stages together form our proposed information- entropy-based multi-step reasoning for large language models, named INFORM. Our experiments across seven reasoning benchmarks utilizing two language models(GPT-3.5-Turbo and text-davinci-003) demonstrate the superiority of INFORM both in performance and efficiency.
The PEACE-Reviews dataset: Modeling Cognitive Appraisals in Emotion Text Analysis

Cognitive appraisal plays a pivotal role in deciphering emotions. Recent studies have delved into its significance, yet the interplay between various forms of cognitive appraisal and specific emotions, such as joy and anger, remains an area of exploration in consumption contexts. Our research introduces the PEACE-Reviews dataset, a unique compilation of annotated autobiographical accounts where individuals detail their emotional and appraisal experiences during interactions with personally significant products or services. Focusing on the inherent variability in consumer experiences, this dataset offers an in-depth analysis of participants’ psychological traits, their evaluative feedback on purchases, and the resultant emotions. Notably, the PEACE-Reviews dataset encompasses emotion, cognition, individual traits, and demographic data. We also introduce preliminary models that predict certain features based on the autobiographical narratives.
Gradient-based Gradual Pruning for Language-Specific Multilingual Neural Machine Translation

Multilingual neural machine translation (MNMT) offers the convenience of translating between multiple languages with a single model. However, MNMT often suffers from performance degradation in high-resource languages compared to bilingual counterparts. This degradation is commonly attributed to parameter interference, which occurs when parameters are fully shared across all language pairs. In this work, to tackle this issue we propose a gradient-based gradual pruning technique for MNMT. Our approach aims to identify an optimal sub-network for each language pair within the multilingual model by leveraging gradient-based information as pruning criterion and gradually increasing the pruning ratio as schedule. Our approach allows for partial parameter sharing across language pairs to alleviate interference, and each pair preserves its unique parameters to capture language-specific information. Comprehensive experiments on IWSLT and WMT datasets show that our approach yields a notable performance gain on both datasets.
Abstractive Open Information Extraction

Open Information Extraction (OpenIE) is a traditional NLP task that extracts structured information from unstructured text to be used for other downstream applications. Traditionally, OpenIE focuses on extracting the surface forms of relations as they appear in the raw text, which we term extractive OpenIE. One of the main drawbacks of this approach is that implicit semantic relations (inferred relations) can not be extracted, compromising the performance of downstream applications. In this paper, we broaden the scope of OpenIE relations from merely the surface form of relations to include inferred relations, which we term abstractive OpenIE. This new task calls for the development of a new abstractive OpenIE training dataset and a baseline neural model that can extract those inferred relations. We also demonstrate the necessity for a new semantics-based metric for evaluating abstractive OpenIE extractions. Via a case study on Complex QA, we demonstrate the effectiveness of abstractive OpenIE.
CRAB: Assessing the Strength of Causal Relationships Between Real-world Events

Understanding narratives requires reasoning about the cause-and-effect relationships between events mentioned in the text. While existing foundation models yield impressive results in many NLP tasks requiring reasoning, it is unclear whether they understand the complexity of the underlying network of causal relationships of events in narratives. In this work, we present CRAB, a new Causal Reasoning Assessment Benchmark designed to evaluate causal understanding of events in real-world narratives. CRAB contains fine-grained, contextual causality annotations for ~2.7K pairs of real-world events that describe various newsworthy event timelines (e.g., the acquisition of Twitter by Elon Musk). Using CRAB, we measure the performance of several large language models, demonstrating that most systems achieve poor performance on the task. Motivated by classical causal principles, we also analyze the causal structures of groups of events in CRAB, and find that models perform worse on causal reasoning when events are derived from complex causal structures compared to simple linear causal chains. We make our dataset and code available to the research community.
Learning Knowledge-Enhanced Contextual Language Representations for Domain Natural Language Understanding

Taolin Zhang, Ruyao Xu, Chengyu Wang, Zhongjie Duan, Cen Chen, Minghui Qiu, Dawei Cheng, Xiaofeng He, Weining Qian. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
Finding Authentic Counterhate Arguments: A Case Study with Public Figures

We explore authentic counterhate arguments for online hateful content toward individuals. Previous efforts are limited to counterhate to fight against hateful content toward groups. Thus, we present a corpus of 54,816 hateful tweet-paragraph pairs, where the paragraphs are candidate counterhate arguments. The counterhate arguments are retrieved from 2,500 online articles from multiple sources. We propose a methodology that assures the authenticity of the counter argument and its specificity to the individual of interest. We show that finding arguments in online articles is an efficient alternative to counterhate generation approaches that may hallucinate unsupported arguments. We also present linguistic insights on the language used in counterhate arguments. Experimental results show promising results. It is more challenging, however, to identify counterhate arguments for hateful content toward individuals not included in the training set.
COMET-M: Reasoning about Multiple Events in Complex Sentences

Understanding the speaker’s intended meaning often involves drawing commonsense inferences to reason about what is not stated explicitly. In multi-event sentences, it requires understanding the relationships between events based on contextual knowledge. We propose COMET-M (Multi-Event), an event-centric commonsense model capable of generating commonsense inferences for a target event within a complex sentence. COMET-M builds upon COMET (Bosselut et al., 2019), which excels at generating event-centric inferences for simple sentences, but struggles with the complexity of multi-event sentences prevalent in natural text. To overcome this limitation, we curate a Multi-Event Inference (MEI) dataset of 35K human-written inferences. We train COMET-M on the human-written inferences and also create baselines using automatically labeled examples. Experimental results demonstrate the significant performance improvement of COMET-M over COMET in generating multi-event inferences. Moreover, COMET-M successfully produces distinct inferences for each target event, taking the complete context into consideration. COMET-M holds promise for downstream tasks involving natural text such as coreference resolution, dialogue, and story understanding.
A Frustratingly Easy Post-Training Quantization Scheme for LLMs

Efficient inference has become crucial for hyper-scale AI models, including large language models, as their parameter count continues to increase for enhanced performance. This necessity holds true regardless of the computing environment, whether it be mobile devices or cloud servers. Quantization emerges as a solution to alleviate the computational burden during inference. By representing models with a reduced bit-width, quantization minimizes the frequency of DRAM access while fully exploiting the parallelism of operations through a dense matrix format. Consequently, quantized models achieve low end-to-end latency and optimize resource utilization by addressing both memory and computing bottlenecks. In this paper, we propose a straightforward post-training quantization scheme, called Z-Fold, that fully utilizes the feature of the Transformer structure widely employed in large language models.
KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning

In task-oriented dialogs (TOD), reinforcement learning (RL) algorithms train a model to directly optimize response for task-related metrics. However, RL often needs to perform exploration, which can be time-consuming due to the slow auto-regressive sequence generation process. We investigate an approach to create a more efficient RL-based algorithm to improve TOD performance in an offline setting. First, we use a faster generation procedure that samples from independent next-word distributions after training the language model (LM) with supervised learning. We then introduce a fine-grained reward function to help the model focus on learning key information in a dialog, by measuring the importance and semantic closeness of each generated token. Experiments on the MultiWoZ dataset show our new training algorithm, Keywords Reinforcement Learning with Next-word Sampling (KRLS), achieves state-of-the-art performance on the end-to-end response generation task, with a 15% training time reduction compared to a standard RL algorithm using auto-regressive generation.
Diversity Enhanced Narrative Question Generation for Storybooks

Question generation (QG) from a given context can enhance comprehension, engagement, assessment, and overall efficacy in learning or conversational environments. Despite recent advancements in QG, the challenge of enhancing or measuring the diversity of generated questions often remains unaddressed. In this paper, we introduce a multi-question generation model (mQG), which is capable of generating multiple, diverse, and answerable questions by focusing on context and questions. To validate the answerability of the generated questions, we employ a SQuAD 2.0 fine-tuned question answering model, classifying the questions as answerable or not. We train and evaluate mQG on the FairytaleQA dataset, a well-structured QA dataset based on storybooks, with narrative questions. We further apply a zero-shot adaptation on the TellMeWhy and SQuAD1.1 datasets. mQG shows promising results across various evaluation metrics, among strong baselines.
CONTRASTE: Supervised Contrastive Pre-training With Aspect-based Prompts For Aspect Sentiment Triplet Extraction

Existing works on Aspect Sentiment Triplet Extraction (ASTE) explicitly focus on developing more efficient fine-tuning techniques for the task. Instead, our motivation is to come up with a generic approach that can improve the downstream performances of multiple ABSA tasks simultaneously. Towards this, we present CONTRASTE, a novel pre-training strategy using CONTRastive learning to enhance the ASTE performance. While we primarily focus on ASTE, we also demonstrate the advantage of our proposed technique on other ABSA tasks such as ACOS, TASD, and AESC. Given a sentence and its associated (aspect, opinion, sentiment) triplets, first, we design aspect-based prompts with corresponding sentiments masked. We then (pre)train an encoder-decoder model by applying contrastive learning on the decoder-generated aspect-aware sentiment representations of the masked terms. For fine-tuning the model weights thus obtained, we then propose a novel multi-task approach where the base encoder-decoder model is combined with two complementary modules, a tagging-based Opinion Term Detector, and a regression-based Triplet Count Estimator. Exhaustive experiments on four benchmark datasets and a detailed ablation study establish the importance of each of our proposed components as we achieve new state-of-the-art ASTE results.
NarrativeXL: a Large-scale Dataset for Long-Term Memory Models

We propose a new large-scale (nearly a million questions) ultra-long-context (more than 50,000 words average document length) reading comprehension dataset. Using GPT 3.5, we summarized each scene in 1,500 hand-curated fiction books from Project Gutenberg, which resulted in approximately 150 scene-level summaries per book. After that, we created a number of reading comprehension questions based on these summaries, including three types of multiple-choice scene recognition questions, as well as free-form narrative reconstruction questions. With 990,595 total questions, our dataset is an order of magnitude larger than the closest alternatives. Crucially, most questions have a known “retention demand”, indicating how long-term of a memory is needed to answer them, which should aid long-term memory performance evaluation. We validate our data in four small-scale experiments: one with human labelers, and three with existing language models. We show that our questions 1) adequately represent the source material 2) can be used to diagnose a model’s memory capacity 3) are not trivial for modern language models even when the memory demand does not exceed those models’ context lengths. Lastly, we provide our code which can be used to further expand the dataset with minimal human labor.
Failures Pave the Way: Enhancing Large Language Models through Tuning-free Rule Accumulation

Large Language Models (LLMs) have showcased impressive performance. However, due to their inability to capture relationships among samples, these frozen LLMs inevitably keep repeating similar mistakes. In this work, we propose our Tuning-free Rule Accumulation (TRAN) framework, which guides LLMs in improving their performance by learning from previous mistakes. Considering data arrives sequentially, LLMs gradually accumulate rules from incorrect cases, forming a rule collection. These rules are then utilized by the LLMs to avoid making similar mistakes when processing subsequent inputs. Moreover, the rules remain independent of the primary prompts, seamlessly complementing prompt design strategies. Experimentally, we show that TRAN improves over recent baselines by a large margin.
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions

End-to-end task-oriented dialogue (EToD) can directly generate responses in an end-to-end fashion without modular training, which attracts escalating popularity. The advancement of deep neural networks, especially the successful use of large pre-trained models, has further led to significant progress in EToD research in recent years. In this paper, we present a thorough review and provide a unified perspective to summarize existing approaches as well as recent trends to advance the development of EToD research. The contributions of this paper can be summarized: (1) First survey: to our knowledge, we take the first step to present a thorough survey of this research field; (2) New taxonomy: we first introduce a unified perspective for EToD, including (i) Modularly EToD and (ii) Fully EToD; (3) New Frontiers: we discuss some potential frontier areas as well as the corresponding challenges, hoping to spur breakthrough research in EToD field; (4) Abundant resources: we build a public website, where EToD researchers could directly access the recent progress. We hope this work can serve as a thorough reference for the EToD research community.
Enhancing Security and Privacy of Patient Data in Healthcare: A SmartPLS Analysis of Blockchain Technology Implementation

Security and privacy of patient data are critical concerns in the healthcare system. This research aims to investigate the impact of implementing blockchain technology on enhancing the security and privacy of patient data in healthcare. Using the SmartPLS analysis method, this study empirically examines the relationships between blockchain technology implementation and data security and data privacy within the healthcare system. The research sample consists of healthcare organizations that have implemented blockchain technology for data management. Data is collected through surveys and analyzed using SmartPLS to assess the effects of blockchain technology on data security and privacy. The findings reveal the positive influence of blockchain technology implementation on enhancing the security and privacy of patient data. The study also identifies challenges, such as scalability and interoperability, that need to be addressed for successful implementation. This research contributes to the existing literature by providing empirical evidence on the benefits and challenges of implementing blockchain technology to safeguard patient data in healthcare systems.
ANALISIS PERUBAHAN PENGGUNAAN LAHAN DI DAS BATANG TABIR KABUPATEN MERANGIN, PROVINSI JAMBI (Land Use Change Analysis in Batang Tabir Watershed Merangin Regency, Jambi Province)

A continuous land use change in a watershed will affect the hydrological characteristics of the watershed. The Batang Tabir watershed has experienced a decrease in its carrying capacity shown by an increase in the frequency of flooding and the area of flood inundation. This study analyzes the land use changes in the Batang Tabir watershed, Merangin Regency, Jambi Province. The data used are DEM, topographic maps, river network maps, and Landsat imagery. Analysis of land use change used the Maximum Likelihood Classification (MLC) and the accuracy of image interpretation was carried out through the Kappa accuracy test. The land use prediction using the ANN model. The results showed that the rate of land use change increased in secondary forests, built-up areas, agriculture, and open areas; primary forests, paddy fields, and shrubs continued to decline. It is predicted in the year 2030, the trend will be the same; the built-up areas, agriculture, and secondary forests will increase, while primary forests, paddy fields, shrubs, and open areas will decrease.
An Efficient Source of Polarization-Entangled Photon-Pairs at Telecom Wavelength

We report an efficient source of polarization-entangled photon-pairs at telecom wavelength using a fiber-coupled MgO:PPLN ridge waveguide. The maximum coincidence-to-accidental ratio (CAR) is ∼ 446 and the generated Φ + Bell-state has ∼ 90% fidelity.
Consistent Learning-Based Breast Tumor Segmentation and Its Application in Sentinel Lymph Node Metastasis Prediction

Accurate staging of lymph nodes provides crucial diagnostic information for breast cancer patients, where segmentation is of great importance by localizing and visualizing the breast tumor of interest. Nevertheless, current segmentation methods perform average when facing large span of tumor sizes, degraded image quality, blurred tumor boundaries, and resulting noise during manual annotation. Therefore, we develop a Multi-scale RepVGG-based Segmentation Network (MPSegNet) to segment breast tumor from MR images. In particular, we construct a consistent learning framework for the MPSegNet to alleviate the impact of noisy labels upon segmentation results. The rationale is that different views covering the same breast tumors are supposed to generate identical segmentation predictions. Then, we predict SLN metastasis given segmented breast tumors, where we evaluate the relationships between the predictive performance and tumor segmentations under different consistencies. The results show the superiority of our method over other state-of-the-art methods. A high consistency among multiple views can boost the segmentation performance during consistent learning. However, the optimal segmentation does not produce the best SLN metastatic prediction results, implying that the dependence of classification upon segmentation needs to be elaborately investigated further.Clinical Relevance— This study facilitates more accurate segmentation of breast tumors with consistent learning, and provides an initial analysis between tumor segmentation and subsequent prediction of SLN metastasis, which has potential significance for the precise medical care of breast cancer patients.
Machine Learning Training in Silicon Photonic Circuits

Training photonic circuits to learn specific machine learning tasks is the key for photonic AI. We review our work on on-chip training in silicon photonic circuits to implement novel classification computing and its recent progress.
Arguing About Choosing a Normative System: Conflict of Laws

This paper presents a formal model of specific reasoning patterns in conflict of laws (CoL). CoL arises when multiple countries have jurisdiction due to the diverse nationalities of the involved factors. When initiating legal action in one country, the question of which country’s substantial law to apply emerges, possibly involving the CoL regulations of other countries (in cases of transmission and renvoi). Moreover, parties contemplating legal action in a case falling under CoL often engage in a deliberation process known as forum shopping: determining which country’s CoL regulations would result in the most favorable outcome for them. Our model integrates deontic logic (specifically Input/Output logic) with proof theory and formal argumentation techniques to model both types of reasoning.
Development of Bilingual Mathematics Digital Pocket Book to Improve Student’s Ability in Understanding Plane Topic

The aim of this study was to develop a valid, practical, and effective bilingual mathematics digital pocket book to improve students’ ability in understanding plane topic. The study was held at SMP Islam Al-Ulum Terpadu Medan. The subject of this research was class VII-A. This study was a research and development (R&amp;D) study using the ADDIE Model. The data collection technique used a questionnaire sheet for the assessment of material experts, teaching materials experts, learning practitioners and student responses. The results showed that the bilingual mathematics digital pocket book in plane topic obtained an average percentage of 93.18% with the criteria "very valid" by the assessment of material experts and teaching materials experts, obtaining an average percentage of 94.5% with the criteria of "very practical" by the assessment. learning practitioners, and obtained an average percentage of 86.72%. with the criteria of "very effective" by student assessment. Bilingual mathematics digital pocket book in plane topic that has been developed in this study was also considered "effective" if it was seen that the post-test scores reached 90% compared to the pre-test scores which reached 56%. Thus, based on the results of the assessment of material experts, teaching material experts, learning practitioners, and students, it can be said that the bilingual mathematics digital pocket book in plane topic has been tested to be valid, practical and effective for use in learning.