GenAI4ELab/papercli-papers huggingface dataset:
MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models
Authors: Dingyao Yu; Kaitao Song; Peiling Lu; Tianyu He; Xu Tan; Wei Ye; Shikun Zhang; Jiang Bian
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: AI-empowered music processing is a diverse feld that encompasses dozens of tasks, ranging from generation tasks (e.g., timbre synthesis) to comprehension tasks (e.g., music classifcation). For developers and amateurs, it is very diffcult to grasp all of these task to satisfy their requirements in music processing, especially considering the huge differences in the representations of music data and the model applicability across platforms among various tasks. Consequently, it is necessary to build a system to organize and integrate these tasks, and thus help practitioners to automatically analyze their demand and call suitable tools as solutions to fulfill their requirements. Inspired by the recent success of large language models (LLMs) in task automation, we develop a system, named MusicAgent, which integrates numerous music-related tools and an autonomous workflow to address user requirements. More specifically, we build 1) toolset that collects tools from diverse sources, including Hugging Face, GitHub, and Web API, etc. 2) an autonomous workflow empowered by LLMs (e.g., ChatGPT) to organize these tools and automatically decompose user requests into multiple sub-tasks and invoke corresponding music tools. The primary goal of this system is to free users from the intricacies of AI-music tools, enabling them to concentrate on the creative aspect. By granting users the freedom to effortlessly combine tools, the system offers a seamless and enriching music experience. The code is available on GitHub along with a brief instructional video.
SentAlign: Accurate and Scalable Sentence Alignment
Authors: Steinthor Steingrimsson; Hrafn Loftsson; Andy Way
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: We present SentAlign, an accurate sentence alignment tool designed to handle very large parallel document pairs. Given user-defined parameters, the alignment algorithm evaluates all possible alignment paths in fairly large documents of thousands of sentences and uses a divide-and-conquer approach to align documents containing tens of thousands of sentences. The scoring function is based on LaBSE bilingual sentence representations. SentAlign outperforms five other sentence alignment tools when evaluated on two different evaluation sets, German-French and English-Icelandic, and on a downstream machine translation task.
QACheck: A Demonstration System for Question-Guided Multi-Hop Fact-Checking
Authors: Liangming Pan; Xinyuan Lu; Min-Yen Kan; Preslav Nakov
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Fact-checking real-world claims often requires intricate, multi-step reasoning due to the absence of direct evidence to support or refute them. However, existing fact-checking systems often lack transparency in their decision-making, making it challenging for users to comprehend their reasoning process. To address this, we propose the Question-guided Multi-hop Fact-Checking (QACheck) system, which guides the model’s reasoning process by asking a series of questions critical for verifying a claim. QACheck has five key modules: a claim verifier, a question generator, a question-answering module, a QA validator, and a reasoner. Users can input a claim into QACheck, which then predicts its veracity and provides a comprehensive report detailing its reasoning process, guided by a sequence of (question, answer) pairs. QACheck also provides the source of evidence supporting each question, fostering a transparent, explainable, and user-friendly fact-checking process.
RobustQA: A Framework for Adversarial Text Generation Analysis on Question Answering Systems
Authors: Yasaman Boreshban; Seyed Morteza Mirbostani; Seyedeh Fatemeh Ahmadi; Gita Shojaee; Fatemeh Kamani; Gholamreza Ghassem-Sani; Seyed Abolghasem Mirroshandel
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Question answering (QA) systems have reached human-level accuracy; however, these systems are not robust enough and are vulnerable to adversarial examples. Recently, adversarial attacks have been widely investigated in text classification. However, there have been few research efforts on this topic in QA. In this article, we have modified the attack algorithms widely used in text classification to fit those algorithms for QA systems. We have evaluated the impact of various attack methods on QA systems at character, word, and sentence levels. Furthermore, we have developed a new framework, named RobustQA, as the first open-source toolkit for investigating textual adversarial attacks in QA systems. RobustQA consists of seven modules: Tokenizer, Victim Model, Goals, Metrics, Attacker, Attack Selector, and Evaluator. It currently supports six different attack algorithms. Furthermore, the framework simplifies the development of new attack algorithms in QA. The source code and documentation of RobustQA are available at https://github.com/mirbostani/RobustQA.
Kandinsky: An Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion
Authors: Anton Razzhigaev; Arseniy Shakhmatov; Anastasia Maltseva; Vladimir Arkhipkin; Igor Pavlov; Ilya Ryabov; Angelina Kuts; Alexander Panchenko; Andrey Kuznetsov; Denis Dimitrov
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Text-to-image generation is a significant domain in modern computer vision and achieved substantial improvements through the evolution of generative architectures. Among these, diffusion-based models demonstrated essential quality enhancements. These models generally split into two categories: pixel-level and latent-level approaches. We present Kandinsky – a novel exploration of latent diffusion architecture, combining the principles of image prior models with latent diffusion techniques. The image prior model, is trained separately to map CLIP text and image embeddings. Another distinct feature of the proposed model is the modified MoVQ implementation, which serves as the image autoencoder component. Overall the designed model contains 3.3B parameters. We also deployed a user-friendly demo system that supports diverse generative modes such as text-to-image generation, image fusion, text and image fusion, image variations generation and text-guided inpainting/outpainting. Additionally we released the source code and checkpoints for Kandinsky models. Experimental evaluations demonstrate FID score of 8.03 on the COCO-30K dataset, marking our model as the top open source performer in terms of measurable image generation quality.
NewsRecLib: A PyTorch-Lightning Library for Neural News Recommendation
Authors: Andreea Iana; Goran Glavaš; Heiko Paulheim
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: NewsRecLib is an open-source library based on Pytorch-Lightning and Hydra developed for training and evaluating neural news recommendation models. The foremost goals of NewsRecLib are to promote reproducible research and rigorous experimental evaluation by (i) providing a unified and highly configurable framework for exhaustive experimental studies and (ii) enabling a thorough analysis of the performance contribution of different model architecture components and training regimes. NewsRecLib is highly modular, allows specifying experiments in a single configuration file, and includes extensive logging facilities. Moreover, NewsRecLib provides out-of-the-box implementations of several prominent neural models, training methods, standard evaluation benchmarks, and evaluation metrics for news recommendation.
MiniChain: A Small Library for Coding with Large Language Models
Authors: Alexander Rush
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Programming augmented by large language models (LLMs) opens up many new application areas, but also requires care. LLMs are accurate enough, on average, to replace core functionality, yet make basic mistakes that demonstrate a lack of robustness. An ecosystem of prompting tools, from intelligent agents to new programming languages, have emerged with different solutions for patching LLMs with other tools. In this work, we introduce MiniChain, an opinionated tool for LLM augmented programming, with the design goals of ease-of-use of prototyping, transparency through automatic visualization, and a minimalistic approach to advanced features. The MiniChain library provides core primitives for coding LLM calls, separating out prompt templates, and capturing program structure. The library includes demo implementations of the main applications papers in the area, including chat-bots, code generation, retrieval-based question answering, and complex information extraction. The library is open-source and available at https://github.com/srush/MiniChain, with code demos available at https://srush-minichain.hf.space/, and video demo at https://www.youtube.com/watch?v=VszZ1VnO7sk.
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback
Authors: Viet Lai; Chien Nguyen; Nghia Ngo; Thuat Nguyen; Franck Dernoncourt; Ryan Rossi; Thien Nguyen
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: A key technology for large language models (LLMs) involves instruction tuning that helps align the models’ responses with human expectations to realize impressive learning abilities. Two major approaches for instruction tuning characterize supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), which are applied to produce the best commercial LLMs. To improve the accessibility of LLMs, various instruction-tuned open-source LLMs have also been introduced recently. However, existing open-source LLMs have only been instruction-tuned for English and a few popular languages, thus hindering their accessibility to many other languages in the world. In addition, SFT has been used as the only approach to instruction-tune open-source LLMs for multiple languages. This has left a significant gap for fine-tuned LLMs based on RLHF in diverse languages and raised important questions on how RLHF can boost the performance of multilingual instruction tuning. To overcome this issue, we present Okapi, the first system with instruction-tuned LLMs based on RLHF for multiple languages. Okapi introduces instruction and response-ranked data in 26 diverse languages to facilitate the experiments and development of future multilingual LLM research. We also present benchmark datasets to enable the evaluation of generative LLMs in multiple languages. Our experiments demonstrate the advantages of RLHF for multilingual instruction over SFT for different base models and datasets. Our framework with created resources, fine-tuned LLMs, interaction scripts are released at https://github.com/nlp-uoregon/Okapi. A demo video to show our framework can also be found at: https://youtu.be/QFV2fkPwvi0.
SAGEViz: SchemA GEneration and Visualization
Authors: Sugam Devare; Mahnaz Koupaee; Gautham Gunapati; Sayontan Ghosh; Sai Vallurupalli; Yash Kumar Lal; Francis Ferraro; Nathanael Chambers; Greg Durrett; Raymond Mooney; Katrin Erk; Niranjan Balasubramanian
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Schema induction involves creating a graph representation depicting how events unfold in a scenario. We present SAGEViz, an intuitive and modular tool that utilizes human-AI collaboration to create and update complex schema graphs efficiently, where multiple annotators (humans and models) can work simultaneously on a schema graph from any domain. The tool consists of two components: (1) a curation component powered by plug-and-play event language models to create and expand event sequences while human annotators validate and enrich the sequences to build complex hierarchical schemas, and (2) an easy-to-use visualization component to visualize schemas at varying levels of hierarchy. Using supervised and few-shot approaches, our event language models can continually predict relevant events starting from a seed event. We conduct a user study and show that users need less effort in terms of interaction steps with SAGEViz to generate schemas of better quality. We also include a video demonstrating the system.
Thresh: A Unified, Customizable and Deployable Platform for Fine-Grained Text Evaluation
Authors: David Heineman; Yao Dou; Wei Xu
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Fine-grained, span-level human evaluation has emerged as a reliable and robust method for evaluating text generation tasks such as summarization, simplification, machine translation and news generation, and the derived annotations have been useful for training automatic metrics and improving language models. However, existing annotation tools implemented for these evaluation frameworks lack the adaptability to be extended to different domains or languages, or modify annotation settings according to user needs; and, the absence of a unified annotated data format inhibits the research in multi-task learning. In this paper, we introduce Thresh, a unified, customizable and deployable platform for fine-grained evaluation. With a single YAML configuration file, users can build and test an annotation interface for any framework within minutes – all in one web browser window. To facilitate collaboration and sharing, Thresh provides a community hub that hosts a collection of fine-grained frameworks and corresponding annotations made and collected by the community, covering a wide range of NLP tasks. For deployment, Thresh offers multiple options for any scale of annotation projects from small manual inspections to large crowdsourcing ones. Additionally, we introduce a Python library to streamline the entire process from typology design and deployment to annotation processing. Thresh is publicly accessible at https://thresh.tools.
InsightPilot: An LLM-Empowered Automated Data Exploration System
Authors: Pingchuan Ma; Rui Ding; Shuai Wang; Shi Han; Dongmei Zhang
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Exploring data is crucial in data analysis, as it helps users understand and interpret the data more effectively. However, performing effective data exploration requires in-depth knowledge of the dataset, the user intent and expertise in data analysis techniques. Not being familiar with either can create obstacles that make the process time-consuming and overwhelming. To address this issue, we introduce InsightPilot, an LLM (Large Language Model)-based, automated data exploration system designed to simplify the data exploration process. InsightPilot features a set of carefully designed analysis actions that streamline the data exploration process. Given a natural language question, InsightPilot collaborates with the LLM to issue a sequence of analysis actions, explore the data and generate insights. We demonstrate the effectiveness of InsightPilot in a user study and a case study, showing how it can help users gain valuable insights from their datasets.
SynJax: Structured Probability Distributions for JAX
Authors: Miloš Stanojević; Laurent Sartran
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: The development of deep learning software libraries enabled significant progress in the field by allowing users to focus on modeling, while letting the library to take care of the tedious and time-consuming task of optimizing execution for modern hardware accelerators. However, this has benefited only particular types of deep learning models, such as Transformers, whose primitives map easily to the vectorized computation. The models that explicitly account for structured objects, such as trees and segmentations, did not benefit equally because they require custom algorithms that are difficult to implement in a vectorized form. SynJax directly addresses this problem by providing an efficient vectorized implementation of inference algorithms for structured distributions covering alignment, tagging, segmentation, constituency trees and spanning trees. This is done by exploiting the connection between algorithms for automatic differentiation and probabilistic inference. With SynJax we can build large-scale differentiable models that explicitly model structure in the data. The code is available at https://github.com/google-deepmind/synjax
RESIN-EDITOR: A Schema-guided Hierarchical Event Graph Visualizer and Editor
Authors: Khanh Duy Nguyen; Zixuan Zhang; Reece Suchocki; Sha Li; Martha Palmer; Susan Windisch Brown; Jiawei Han; Heng Ji
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: In this paper, we present RESIN-EDITOR, an interactive event graph visualizer and editor designed for analyzing complex events. Our RESIN-EDITOR system allows users to render and freely edit hierarchical event graphs extracted from multimedia and multi-document news clusters with guidance from human-curated event schemas. RESIN-EDITOR’s unique features include hierarchical graph visualization, comprehensive source tracing, and interactive user editing, which significantly outperforms existing Information Extraction (IE) visualization tools in both IE result analysis and general model improvements. In our evaluation of RESIN-EDITOR, we demonstrate ways in which our tool is effective in understanding complex events and enhancing system performances. The source code, a video demonstration, and a live website for RESIN-EDITOR have been made publicly available.
DRGCoder: Explainable Clinical Coding for the Early Prediction of Diagnostic-Related Groups
Authors: Daniel Hajialigol; Derek Kaknes; Tanner Barbour; Daphne Yao; Chris North; Jimeng Sun; David Liem; Xuan Wang
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Medical claim coding is the process of transforming medical records, usually presented as free texts written by clinicians, or discharge summaries, into structured codes in a classification system such as ICD-10 (International Classification of Diseases, Tenth Revision) or DRG (Diagnosis-Related Group) codes. This process is essential for medical billing and transitional care; however, manual coding is time-consuming, error-prone, and expensive. To solve these issues, we propose DRGCoder, an explainability-enhanced clinical claim coding system for the early prediction of medical severity DRGs (MS-DRGs), a classification system that categorizes patients’ hospital stays into various DRG groups based on the severity of illness and mortality risk. The DRGCoder framework introduces a novel multi-task Transformer model for MS-DRG prediction, modeling both the DRG labels of the discharge summaries and the important, or salient words within he discharge summaries. We allow users to inspect DRGCoder’s reasoning by visualizing the weights for each word of the input. Additionally, DRGCoder allows users to identify diseases within discharge summaries and compare across multiple discharge summaries. Our demo is available at https://huggingface.co/spaces/danielhajialigol/DRGCoder. A video demonstrating the demo can be found at https://www.youtube.com/watch?v=pcdiG6VwqlA
CAMRA: Copilot for AMR Annotation
Authors: Jon Z. Cai; Shafiuddin Rehan Ahmed; Julia Bonn; Kristin Wright-Bettner; Martha Palmer; James H. Martin
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: In this paper, we introduce CAMRA (Copilot for AMR Annotatations), a cutting-edge web-based tool designed for constructing Abstract Meaning Representation (AMR) from natural language text. CAMRA offers a novel approach to deep lexical semantics annotation such as AMR, treating AMR annotation akin to coding in programming languages. Leveraging the familiarity of programming paradigms, CAMRA encompasses all essential features of existing AMR editors, including example lookup, while going a step further by integrating Propbank roleset lookup as an autocomplete feature within the tool. Notably, CAMRA incorporates AMR parser models as coding co-pilots, greatly enhancing the efficiency and accuracy of AMR annotators.
Reaction Miner: An Integrated System for Chemical Reaction Extraction from Textual Data
Authors: Ming Zhong; Siru Ouyang; Yizhu Jiao; Priyanka Kargupta; Leo Luo; Yanzhen Shen; Bobby Zhou; Xianrui Zhong; Xuan Liu; Hongxiang Li; Jinfeng Xiao; Minhao Jiang; Vivian Hu; Xuan Wang; Heng Ji; Martin Burke; Huimin Zhao; Jiawei Han
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Chemical reactions, as a core entity in the realm of chemistry, hold crucial implications in diverse areas ranging from hands-on laboratory research to advanced computational drug design. Despite a burgeoning interest in employing NLP techniques to extract these reactions, aligning this task with the real-world requirements of chemistry practitioners remains an ongoing challenge. In this paper, we present Reaction Miner, a system specifically designed to interact with raw scientific literature, delivering precise and more informative chemical reactions. Going beyond mere extraction, Reaction Miner integrates a holistic workflow: it accepts PDF files as input, bypassing the need for pre-processing and bolstering user accessibility. Subsequently, a text segmentation module ensures that the refined text encapsulates complete chemical reactions, augmenting the accuracy of extraction. Moreover, Reaction Miner broadens the scope of existing pre-defined reaction roles, including vital attributes previously neglected, thereby offering a more comprehensive depiction of chemical reactions. Evaluations conducted by chemistry domain users highlight the efficacy of each module in our system, demonstrating Reaction Miner as a powerful tool in this field.
CHAMP: Efficient Annotation and Consolidation of Cluster Hierarchies
Authors: Arie Cattan; Tom Hope; Doug Downey; Roy Bar-Haim; Lilach Eden; Yoav Kantor; Ido Dagan
Venue: EMNLP 2023 (Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations)

Abstract: Various NLP tasks require a complex hierarchical structure over nodes, where each node is a cluster of items. Examples include generating entailment graphs, hierarchical cross-document coreference resolution, annotating event and subevent relations, etc. To enable efficient annotation of such hierarchical structures, we release CHAMP, an open source tool allowing to incrementally construct both clusters and hierarchy simultaneously over any type of texts. This incremental approach significantly reduces annotation time compared to the common pairwise annotation approach and also guarantees maintaining transitivity at the cluster and hierarchy levels. Furthermore, CHAMP includes a consolidation mode, where an adjudicator can easily compare multiple cluster hierarchy annotations and resolve disagreements.