Reviewerly/exHarmony huggingface dataset:
LHS712EE at BioLaySumm 2023: Using BART and LED to summarize biomedical research articles

As part of our participation in BioLaySumm 2023, we explored the use of large language models (LLMs) to automatically generate concise and readable summaries of biomedical research articles.We utilized pre-trained LLMs to fine-tune our summarization models on two provided datasets, and adapt them to the shared task within the constraints of training time and computational power.Our final models achieved very high relevance and factuality scores on the test set, and ranked among the top five models in the overall performance.
Few-Shot Document-Level Event Argument Extraction

Event argument extraction (EAE) has been well studied at the sentence level but under-explored at the document level.In this paper, we study to capture event arguments that actually spread across sentences in documents.Prior works usually assume full access to rich document supervision, ignoring the fact that the available argument annotation is usually limited.To fill this gap, we present FewDocAE, a Few-Shot Document-Level Event Argument Extraction benchmark, based on the existing documentlevel event extraction dataset.We first define the new problem and reconstruct the corpus by a novel N -Way-D-Doc sampling instead of the traditional N -Way-K-Shot strategy.Then we adjust the current document-level neural models into the few-shot setting to provide baseline results under in-and cross-domain settings.Since the argument extraction depends on the context from multiple sentences and the learning process is limited to very few examples, we find this novel task to be very challenging with substantively low performance.Considering FewDocAE is closely related to practical use under low-resource regimes, we hope this benchmark encourages more research in this direction.Our data and codes will be available online 1 .
Graph-based Relation Mining for Context-free Out-of-vocabulary Word Embedding Learning

The out-of-vocabulary (OOV) words are difficult to represent while critical to the performance of embedding-based downstream models.Prior OOV word embedding learning methods failed to model complex word formation well.In this paper, we propose a novel graph-based relation mining method, namely GRM, for OOV word embedding learning.We first build a Word Relationship Graph (WRG) based on word formation and associate OOV words with their semantically relevant words, which can mine the relational information inside word structures.Subsequently, our GRM can infer high-quality embeddings for OOV words through passing and aggregating semantic attributes and relational information in the WRG, regardless of contextual richness.Extensive experiments demonstrate that our model significantly outperforms state-of-theart baselines on both intrinsic and downstream tasks when faced with OOV words.
Teacher and Student Models of Offensive Language in Social Media

State-of-the-art approaches to identifying offensive language online make use of large pretrained transformer models.However, the inference time, disk, and memory requirements of these transformer models present challenges for their wide usage in the real world.Even the distilled transformer models remain prohibitively large for many usage scenarios.To cope with these challenges, in this paper, we propose transferring knowledge from transformer models to much smaller neural models to make predictions at the token-and at the post-level.We show that this approach leads to lightweight offensive language identification models that perform on par with large transformers but with 100 times fewer parameters and much less memory usage.
SheffieldVeraAI at SemEval-2023 Task 3: Mono and Multilingual Approaches for News Genre, Topic and Persuasion Technique Classification

This paper describes our approach for SemEval-2023 Task 3: Detecting the category, the framing, and the persuasion techniques in online news in a multilingual setup.For Subtask 1 (News Genre), we propose an ensemble of fully trained and adapter mBERT models which was ranked joint-first for German, and had the highest mean rank of multi-language teams.For Subtask 2 (Framing), we achieved first place in 3 languages, and the best average rank across all the languages, by using two separate ensembles: a monolingual RoBERTa-MUPPET LARGE and an ensemble of XLM-RoBERTa LARGE with adapters and task adaptive pretraining.For Subtask 3 (Persuasion Techniques), we trained a monolingual RoBERTa-Base model for English and a multilingual mBERT model for the remaining languages, which achieved top 10 for all languages, including 2nd for English.For each subtask, we compared monolingual and multilingual approaches, and considered class imbalance techniques. 1
Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks

State-of-the-art models can perform well in controlled environments, but they often struggle when presented with out-of-distribution (OOD) examples, making OOD detection a critical component of NLP systems.In this paper, we focus on highlighting the limitations of existing approaches to OOD detection in NLP.Specifically, we evaluated eight OOD detection methods that are easily integrable into existing NLP systems and require no additional OOD data or model modifications.One of our contributions is providing a well-structured research environment that allows for full reproducibility of the results.Additionally, our analysis shows that existing OOD detection methods for NLP tasks are not yet sufficiently sensitive to capture all samples characterized by various types of distributional shifts.Particularly challenging testing scenarios arise in cases of background shift and randomly shuffled word order within in domain texts.This highlights the need for future work to develop more effective OOD detection approaches for the NLP problems, and our work provides a well-defined foundation for further research in this area.
Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback

Frozen models trained to mimic static datasets can never improve their performance.Models that can employ internet-retrieval for up-to-date information and obtain feedback from humans during deployment provide the promise of both adapting to new information, and improving their performance.In this work we study how to improve internet-driven conversational skills in such a learning framework.We collect deployment data, which we make publicly available, of human interactions, and collect various types of human feedback -including binary quality measurements, free-form text feedback, and fine-grained reasons for failure.We then study various algorithms for improving from such feedback, including standard supervised learning, rejection sampling, modelguiding and reward-based learning, in order to make recommendations on which type of feedback and algorithms work best.We find the recently introduced DIRECTOR model (Arora et al., 2022) shows significant improvements over other existing approaches.
Holistic Prediction on a Time-Evolving Attributed Graph

Graph-based prediction is essential in NLP tasks such as temporal knowledge graph completion.A cardinal question in this field is, how to predict the future links, nodes, and attributes of a time-evolving attributed graph?Unfortunately, existing techniques assume that each link, node, and attribute prediction is independent, and fall short of predicting the appearance of new nodes that were not observed in the past.In this paper, we address two interrelated questions; (1) can we exploit task interdependence to improve prediction accuracy?and (2) can we predict new nodes with their attributes?We propose a unified framework that predicts node attributes and topology changes such as the appearance and disappearance of links and the emergence and loss of nodes.This framework comprises components for independent and interactive prediction and for predicting new nodes.Our experimental study using realworld data confirms that our interdependent prediction framework achieves higher accuracy than methods based on independent prediction.
Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems

Dialog systems are often designed or trained to output human-like responses.However, some responses may be impossible for a machine to truthfully say (e.g."that movie made me cry").Highly anthropomorphic responses might make users uncomfortable or implicitly deceive them into thinking they are interacting with a human.We collect human ratings on the feasibility of approximately 900 two-turn dialogs sampled from 9 diverse data sources.Ratings are for two hypothetical machine embodiments: a futuristic humanoid robot and a digital assistant.We find that for some data-sources commonly used to train dialog systems, 20-30% of utterances are not viewed as possible for a machine.Rating is marginally affected by machine embodiment.We explore qualitative and quantitative reasons for these ratings.Finally, we build classifiers and explore how modeling configuration might affect output permissibly, and discuss implications for building less falsely anthropomorphic dialog systems.
Using Commonsense Knowledge to Answer Why-Questions

Answering questions in narratives about why events happened often requires commonsense knowledge external to the text.What aspects of this knowledge are available in large language models?What aspects can be made accessible via external commonsense resources?We study these questions in the context of answering questions in the TELLMEWHY dataset using COMET as a source of relevant commonsense relations.We analyze the effects of model size (T5 variants and GPT-3) along with methods of injecting knowledge (COMET) into these models.Results show that the largest models, as expected, yield substantial improvements over base models and injecting external knowledge helps models of all sizes.We also find that the format in which knowledge is provided is critical, and that smaller models benefit more from larger amounts of knowledge.Finally, we develop an ontology of knowledge types and analyze the relative coverage of the models across these categories.
Handling and Presenting Harmful Text in NLP Research

Text data can pose a risk of harm.However, the risks are not fully understood, and how to handle, present, and discuss harmful text in a safe way remains an unresolved issue in the NLP community.We provide an analytical framework categorising harms on three axes:(1) the harm type (e.g., misinformation, hate speech or racial stereotypes); (2) whether a harm is sought as a feature of the research design if explicitly studying harmful content (e.g., training a hate speech classifier), versus unsought if harmful content is encountered when working on unrelated problems (e.g., language generation or part-of-speech tagging); and (3) who it affects, from people (mis)represented in the data to those handling the data and those publishing on the data.We provide advice for practitioners, with concrete steps for mitigating harm in research and in publication.To assist implementation we introduce HARMCHECK -a documentation standard for handling and presenting harmful text in research.
ConNER: Consistency Training for Cross-lingual Named Entity Recognition

Cross-lingual named entity recognition (NER) suffers from data scarcity in the target languages, especially under zero-shot settings.Existing translate-train or knowledge distillation methods attempt to bridge the language gap, but often introduce a high level of noise.To solve this problem, consistency training methods regularize the model to be robust towards perturbations on data or hidden states.However, such methods are likely to violate the consistency hypothesis, or mainly focus on coarse-grain consistency.We propose ConNER as a novel consistency training framework for cross-lingual NER, which comprises of: (1) translation-based consistency training on unlabeled target-language data, and (2) dropoutbased consistency training on labeled sourcelanguage data.ConNER effectively leverages unlabeled target-language data and alleviates overfitting on the source language to enhance the cross-lingual adaptability.Experimental results show our ConNER achieves consistent improvement over various baseline methods. 1
Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems

Numerical Question Answering is the task of answering questions that require numerical capabilities.Previous works introduce general adversarial attacks to Numerical Question Answering, while not systematically exploring numerical capabilities specific to the topic.In this paper, we propose to conduct numerical capability diagnosis on a series of Numerical Question Answering systems and datasets.A series of numerical capabilities are highlighted, and corresponding dataset perturbations are designed.Empirical results indicate that existing systems are severely challenged by these perturbations.E.g., Graph2Tree experienced a 53.83% absolute accuracy drop against the "Extra" perturbation on ASDiv-a, and BART experienced 13.80% accuracy drop against the "Language" perturbation on the numerical subset of DROP.As a counteracting approach, we also investigate the effectiveness of applying perturbations as data augmentation to relieve systems' lack of robust numerical capabilities.With experiment analysis and empirical studies, it is demonstrated that Numerical Question Answering with robust numerical capabilities is still to a large extent an open question.We discuss future directions of Numerical Question Answering and summarize guidelines on future dataset collection and system design.
NarraSum: A Large-Scale Dataset for Abstractive Narrative Summarization

Narrative summarization aims to produce a distilled version of a narrative to describe its most salient events and characters.Summarizing a narrative is challenging as it requires an understanding of event causality and character behaviors.To encourage research in this direction, we propose NARRASUM, a largescale narrative summarization dataset.It contains 122K narrative documents, which are collected from plot descriptions of movies and TV episodes with diverse genres, and their corresponding abstractive summaries.Experiments show that there is a large performance gap between humans and the state-of-the-art summarization models on NARRASUM.We hope that this dataset will promote future research in summarization, as well as broader studies of natural language understanding and generation.The dataset is available at https: //github.com/zhaochaocs/narrasum.
A Simple Contrastive Learning Framework for Interactive Argument Pair Identification via Argument-Context Extraction

Interactive argument pair identification is an emerging research task for argument mining, aiming to identify whether two arguments are interactively related.It is pointed out that the context of the argument is essential to improve identification performance.However, current context-based methods achieve limited improvements since the entire context typically contains much irrelevant information.In this paper, we propose a simple contrastive learning framework to solve this problem by extracting valuable information from the context.This framework can construct hard argumentcontext samples and obtain a robust and uniform representation by introducing contrastive learning.We also propose an argument-context extraction module to enhance information extraction by discarding irrelevant blocks.The experimental results show that our method achieves the state-of-the-art performance on the benchmark dataset.Further analysis demonstrates the effectiveness of our proposed modules and visually displays more compact semantic representations.The code is available at GitHub 1 .
A Dual-branch Learning Model with Gradient-balanced Loss for Long-tailed Multi-label Text Classification

Multi-label text classification has a wide range of applications in the real world. However, the data distribution in the real world is often imbalanced, which leads to serious long-tailed problems. For multi-label classification, due to the vast scale of datasets and existence of label co-occurrence, how to effectively improve the prediction accuracy of tail labels without degrading the overall precision becomes an important challenge. To address this issue, we propose A Dual-Branch Learning Model with Gradient-Balanced Loss (DBGB) based on the paradigm of existing pre-trained multi-label classification SOTA models. Our model consists of two main long-tailed module improvements. First, with the shared text representation, the dual-classifier is leveraged to process two kinds of label distributions; one is the original data distribution and the other is the under-sampling distribution for head labels to strengthen the prediction for tail labels. Second, the proposed gradient-balanced loss can adaptively suppress the negative gradient accumulation problem related to labels, especially tail labels. We perform extensive experiments on three multi-label text classification datasets. The results show that the proposed method achieves competitive performance on overall prediction results compared to the state-of-the-art methods in solving the multi-label classification, with significant improvement on tail-label accuracy.
Chapter 1. The Roles of Symbols in Neural-Based AI: They Are Not What You Think!

We propose that symbols are first and foremost external communication tools used between intelligent agents that allow knowledge to be transferred in a more efficient and effective manner than having to experience the world directly. But, they are also used internally within an agent through a form of self-communication to help formulate, describe and justify subsymbolic patterns of neural activity that truly implement thinking. Symbols, and our languages that make use of them, not only allow us to explain our thinking to others and ourselves, but also provide beneficial constraints (inductive bias) on learning about the world. In this paper we present relevant insights from neuroscience and cognitive science, about how the human brain represents symbols and the concepts they refer to, and how today’s artificial neural networks can do the same. We then present a novel neuro-symbolic hypothesis and a plausible architecture for intelligent agents that combines subsymbolic representations for symbols and concepts for learning and reasoning. Our hypothesis and associated architecture imply that symbols will remain critical to the future of intelligent systems NOT because they are the fundamental building blocks of thought, but because they are characterizations of subsymbolic processes that constitute thought.
Machine Learning Application Benchmark

This paper presents the MLAB project, a research and development activity funded by ESA General Support Technology Programme under the lead of Airbus Defence and Space GmbH, with the goal of developing a machine learning application benchmark for space applications. First, the need for a benchmark dedicated to machine learning applications in spacecraft is explained, and examples of applications are described including their design challenges. Then the benchmark design is presented, including the rules of the metrics, guidelines and scenarios for references. These scenarios include a description of the reference workloads that have been selected during the activity as representative for spacecraft applications. Lastly, the submission concept is introduced.
CoVE

Multi-tenant computing platforms are typically comprised of several software and hardware components including platform firmware, operating system, virtualization monitor, and tenant workloads (typically in a virtual machine, container, or application). This model is well established in large scale commercial deployments, but the downside is that all platform components and operators are in the Trusted Computing Base (TCB) of the tenant. This aspect is ill-suited for privacy-oriented workloads that aim to minimize the TCB. Confidential computing [1] presents a good stepping-stone towards providing a quantifiable TCB for computing. Confidential computing requires the use of a HW-attested Trusted Execution Environment for data-in-use protection. The RISC-V architecture presents a strong foundation for meeting the requirements for Confidential Computing in a clean slate manner. This paper describes a reference architecture and discusses ISA, non-ISA and System-on-Chip (SoC) requirements for confidential computing on RISC-V Platforms. It discusses proposed RISC-V ISA and non-ISA for Confidential Virtual-machine Extension, referred to as CoVE.
Changes in Navigation over Time: A Comparison of Teleportation and Joystick-Based Locomotion

Little research has studied how people use Virtual Reality (VR) changes as they experience VR. This article reports the results of an experiment investigating how users’ behavior with two locomotion methods changed over 4 weeks: teleportation and joystick-based locomotion. Twenty novice VR users (with no more than 1 hour prior experience with any form of walking in VR) were recruited. They loaned an Oculus Quest for 4 weeks on their own time, including an activity we provided them with. Results showed that the time required to complete the navigation task decreased faster for joystick-based locomotion. Spatial memory improved with time, particularly when using teleportation (which starts disadvantaged to joystick-based locomotion). In addition, overall cybersickness decreased slightly over time; however, two dimensions of cybersickness (nausea and disorientation) increased notably over time using joystick-based navigation.
A Comparative Study on Transformer-based News Summarization

News articles play a crucial role in helping humans know about many important events, developments, and inventions worldwide. The busy chores of our day-to-day life have made it quite challenging to consume important information from lengthy news articles. Therefore, short summaries of news articles are not only crucial but essential as well. Deep learning has revolutionized the field of natural language processing research. A lot of research has been done using pre-trained transformer-based models, and it has significantly improved the text sum-marization performance. In this paper, efforts were made to analyze transformer-based models such as BERT, GPT-2, XL Net, BART, and T5 for extractive and abstractive summarizations. This research investigates various methods through observation and experimentation. It also proposes methods that produce better summaries than comparable methods.
SIE-YOLOv5: Improved YOLOv5 for Small Object Detection in Drone-Captured-Scenarios

With the widespread application of unmanned aerial vehicles (UAVs), object detection from the perspective of UAVs has received increasing attention from scholars. The high image density and small objects in images captured by UAVs pose significant challenges for detection. To address this problem, we propose a detection algorithm for UAV images, called SIE-YOLOv5. Based on YOLOv5, we fully explore the downsampling process and propose the Spatial Information Extraction (SIE) structure to fully utilize the information in feature maps, and better detect small objects in images. Then, we improve the feature pyramid pooling structure and propose a new LeakySPPF module, which achieves faster speed while maintaining comparable performance. To better locate attention regions, we incorporate the attention mechanism CBAM [1] into the model. Finally, we also improve the IoU calculation method of the baseline model by introducing Wise-IoU [2] to address calculation issues. Through extensive experiments, we demonstrate that our proposed SIE-YOLOv5 has better small object detection capabilities in UAV-captured scenes. On the VisDrone2021 dataset, the mAP is improved by 6.6%.