Miracle-dz/UniFAR-FTUs huggingface dataset:
Representation learning of writing style

In this paper, we introduce a new method of representation learning that aims to embed documents in a stylometric space. Previous studies in the field of authorship analysis focused on feature engineering techniques in order to represent document styles and to enhance model performance in specific tasks. Instead, we directly embed documents in a stylometric space by relying on a reference set of authors and the intra-author consistency property which is one of two components in our definition of writing style. The main intuition of this paper is that we can define a general stylometric space from a set of reference authors such that, in this space, the coordinates of different documents will be close when the documents are by the same author, and spread away when they are by different authors, even for documents by authors who are not in the set of reference authors. The method we propose allows for the clustering of documents based on stylistic clues reflecting the authorship of documents. For the empirical validation of the method, we train a deep neural network model to predict authors of a large reference dataset consisting of news and blog articles. Albeit the learning process is supervised, it does not require a dedicated labeling of the data but it relies only on the metadata of the articles which are available in huge amounts. We evaluate the model on multiple datasets, on both the authorship clustering and the authorship attribution tasks.
STANDER: An Expert-Annotated Dataset for News Stance Detection and Evidence Retrieval

We present a new challenging news dataset that targets both stance detection (SD) and fine-grained evidence retrieval (ER). With its 3,291 expert-annotated articles, the dataset constitutes a high-quality benchmark for future research in SD and multi-task learning. We provide a detailed description of the corpus collection methodology and carry out an extensive analysis on the sources of disagreement between annotators, observing a correlation between their disagreement and the diffusion of uncertainty around a target in the real world. Our experiments show that the dataset poses a strong challenge to recent state-of-the-art models. Notably, our dataset aligns with an existing Twitter SD dataset: their union thus addresses a key shortcoming of previous works, by providing the first dedicated resource to study multi-genre SD as well as the interplay of signals from social media and news sources in rumour verification.
Let’s Stop Error Propagation in the End-to-End Relation Extraction Literature!

Despite efforts to distinguish three different evaluation setups (Bekoulis et al., 2018), numerous end-to-end Relation Extraction (RE) articles present unreliable performance comparison to previous work. In this paper, we first identify several patterns of invalid comparisons in published papers and describe them to avoid their propagation. We then propose a small empirical study to quantify the impact of the most common mistake and evaluate it leads to overestimating the final RE performance by around 5% on ACE05. We also seize this opportunity to study the unexplored ablations of two recent developments: the use of language model pretraining (specifically BERT) and span-level NER. This meta-analysis emphasizes the need for rigor in the report of both the evaluation setting and the datasets statistics and we call for unifying the evaluation setting in end-to-end RE.
DeepPaperComposer: A Simple Solution for Training Data Preparation for Parsing Research Papers

We present DeepPaperComposer, a simple solution for preparing highly accurate (100%) training data without manual labeling to extract content from scholarly articles using convolutional neural networks (CNNs). We used our approach to generate data and trained CNNs to extract eight categories of both textual (titles, abstracts, headers, figure and table captions, and other texts) and non-textural content (figures and tables) from 30 years of IEEE VIS conference papers, of which a third were scanned bitmap PDFs. We curated this dataset and named it VISpaper-3K. We then showed our initial benchmark performance using VISpaper-3K over itself and CS-150 using YOLOv3 and Faster-RCNN. We open-source DeepPaperComposer of our training data generation and released the resulting annotation data VISpaper-3K to promote re-producible research.
Paraphrase Generation via Adversarial Penalizations

Paraphrase generation is an important problem in Natural Language Processing that has been addressed with neural network-based approaches recently. This paper presents an adversarial framework to address the paraphrase generation problem in English. Unlike previous methods, we employ the discriminator output as penalization instead of using policy gradients, and we propose a global discriminator to avoid the Monte-Carlo search. In addition, this work use and compare different settings of input representation. We compare our methods to some baselines in the Quora question pairs dataset. The results show that our framework is competitive against the previous benchmarks.
Exploring Span Representations in Neural Coreference Resolution

In coreference resolution, span representations play a key role to predict coreference links accurately. We present a thorough examination of the span representation derived by applying BERT on coreference resolution (Joshi et al., 2019) using a probing model. Our results show that the span representation is able to encode a significant amount of coreference information. In addition, we find that the head-finding attention mechanism involved in creating the spans is crucial in encoding coreference knowledge. Last, our analysis shows that the span representation cannot capture non-local coreference as efficiently as local coreference.
Exploring Neural Entity Representations for Semantic Information

Neural methods for embedding entities are typically extrinsically evaluated on downstream tasks and, more recently, intrinsically using probing tasks. Downstream task-based comparisons are often difficult to interpret due to differences in task structure, while probing task evaluations often look at only a few attributes and models. We address both of these issues by evaluating a diverse set of eight neural entity embedding methods on a set of simple probing tasks, demonstrating which methods are able to remember words used to describe entities, learn type, relationship and factual information, and identify how frequently an entity is mentioned. We also compare these methods in a unified framework on two entity linking tasks and discuss how they generalize to different model architectures and datasets.
The Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation

Does neural machine translation yield translations that are congenial with common sense? In this paper, we present a test suite to evaluate the commonsense reasoning capability of neural machine translation. The test suite consists of three test sets, covering lexical and contextless/contextual syntactic ambiguity that requires commonsense knowledge to resolve. We manually create 1,200 triples, each of which contain a source sentence and two contrastive translations, involving 7 different common sense types. Language models pretrained on large-scale corpora, such as BERT, GPT-2, achieve a commonsense reasoning accuracy of lower than 72% on target translations of this test suite. We conduct extensive experiments on the test suite to evaluate commonsense reasoning in neural machine translation and investigate factors that have impact on this capability. Our experiments and analyses demonstrate that neural machine translation performs poorly on commonsense reasoning of the three ambiguity types in terms of both reasoning accuracy ( 6 60.1%) and reasoning consistency (6 31%). We will release our test suite as a machine translation commonsense reasoning testbed to promote future work in this direction.
On the Interplay Between Fine-tuning and Sentence-Level Probing for Linguistic Knowledge in Pre-Trained Transformers

Fine-tuning pre-trained contextualized embedding models has become an integral part of the NLP pipeline. At the same time, probing has emerged as a way to investigate the linguistic knowledge captured by pre-trained models. Very little is, however, understood about how fine-tuning affects the representations of pre-trained models and thereby the linguistic knowledge they encode. This paper contributes towards closing this gap. We study three different pre-trained models: BERT, RoBERTa, and ALBERT, and investigate through sentence-level probing how fine-tuning affects their representations. We find that for some probing tasks fine-tuning leads to substantial changes in accuracy, possibly suggesting that fine-tuning introduces or even removes linguistic knowledge from a pre-trained model. These changes, however, vary greatly across different models, fine-tuning and probing tasks. Our analysis reveals that while fine-tuning indeed changes the representations of a pre-trained model and these changes are typically larger for higher layers, only in very few cases, fine-tuning has a positive effect on probing accuracy that is larger than just using the pre-trained model with a strong pooling method. Based on our findings, we argue that both positive and negative effects of fine-tuning on probing require a careful interpretation.
DivGAN: Towards Diverse Paraphrase Generation via Diversified Generative Adversarial Network

Paraphrases refer to texts that convey the same meaning with different expression forms. Traditional seq2seq-based models on paraphrase generation mainly focus on the fidelity while ignoring the diversity of outputs. In this paper, we propose a deep generative model to generate diverse paraphrases. We build our model based on the conditional generative adversarial network, and propose to incorporate a simple yet effective diversity loss term into the model in order to improve the diversity of outputs. The proposed diversity loss maximizes the ratio of pairwise distance between the generated texts and their corresponding latent codes, forcing the generator to focus more on the latent codes and produce diverse samples. Experimental results on benchmarks of paraphrase generation show that our proposed model can generate more diverse paraphrases compared with baselines.
A comparison between CNNs and WFAs for Sequence Classification

We compare a classical CNN architecture for sequence classification involving several convolutional and max-pooling layers against a simple model based on weighted finite state automata (WFA). Each model has its advantages and disadvantages and it is possible that they could be combined. However, we believe that the first research goal should be to investigate and understand how do these two apparently dissimilar models compare in the context of specific natural language processing tasks. This paper is the first step towards that goal. Our experiments with five sequence classification datasets suggest that, despite the apparent simplicity of WFA models and training algorithms, the performance of WFAs is comparable to that of the CNNs.
Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation

Automated generation of medical reports that describe the findings in the medical images helps radiologists by alleviating their workload. Medical report generation system should generate correct and concise reports. However, data imbalance makes it difficult to train models accurately. Medical datasets are commonly imbalanced in their finding labels because incidence rates differ among diseases; moreover, the ratios of abnormalities to normalities are significantly imbalanced. We propose a novel reinforcement learning method with a reconstructor to improve the clinical correctness of generated reports to train the data-to-text module with a highly imbalanced dataset. Moreover, we introduce a novel data augmentation strategy for reinforcement learning to additionally train the model on infrequent findings. From the perspective of a practical use, we employ a Two-Stage Medical Report Generator (TS-MRGen) for controllable report generation from input images. TS-MRGen consists of two separated stages: an image diagnosis module and a data-to-text module. Radiologists can modify the image diagnosis module results to control the reports that the data-to-text module generates. We conduct an experiment with two medical datasets to assess the data-to-text module and the entire two-stage model. Results demonstrate that the reports generated by our model describe the findings in the input image more correctly.
ConceptBert: Concept-Aware Representation for Visual Question Answering

Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. A VQA model combines visual and textual features in order to answer questions grounded in an image. Current works in VQA focus on questions which are answerable by direct analysis of the question and image alone. We present a concept-aware algorithm, ConceptBert, for questions which require common sense, or basic factual knowledge from external structured content. Given an image and a question in natural language, ConceptBert requires visual elements of the image and a Knowledge Graph (KG) to infer the correct answer. We introduce a multi-modal representation which learns a joint Concept-Vision-Language embedding inspired by the popular BERT architecture. We exploit ConceptNet KG for encoding the common sense knowledge and evaluate our methodology on the Outside Knowledge-VQA (OK-VQA) and VQA datasets.
Durable Mechanical Circulatory Support in Patients With Amyloid Cardiomyopathy

Supplemental Digital Content is available in the text. Background: Many patients with amyloid cardiomyopathy (ACM) develop advanced heart failure, and durable mechanical circulatory support (MCS) may be a consideration. However, data describing clinical outcomes after MCS in this population are limited. Methods: Adult patients in the Interagency Registry for Mechanically Assisted Circulatory Support with dilated cardiomyopathy (DCM, n=19 921), nonamyloid restrictive cardiomyopathy (RCM, n=248), or ACM (n=46) between 2005 and 2017 were included. Patient and device characteristics were compared between cardiomyopathy groups. The primary end point was the cumulative incidence of death with heart transplantation as a competing risk. Results: Patients with ACM (n=46) were older (61 years [interquartile range, 55–69 years] versus 58 years [interquartile range, 49–66 years] for DCM and 55 years [interquartile range, 46–62 years] for nonamyloid RCM, P<0.001) and were more commonly Interagency Registry for Mechanically Assisted Circulatory Support profile 1 (30.4% versus 17.9% for DCM and 21.0% for nonamyloid RCM, P=0.04) at device implantation. Use of biventricular support (biventricular assist device or total artificial heart) was the highest for patients with ACM (41.3% versus 6.7% and 19.4% for patients with DCM and nonamyloid RCM, respectively, P=0.014). The cumulative incidence of death was highest for patients with ACM relative to those with DCM or nonamyloid RCM (P<0.001) but did not differ significantly between groups for those who required biventricular MCS. Conclusions: Compared with patients with DCM or nonamyloid RCM who received durable MCS, those with ACM experienced the highest use of biventricular support and the worst survival. These data highlight concerns with the use of durable MCS for patients with ACM.
Hybrid emergency room shows maximum effect on trauma resuscitation when used in patients with higher severity.

BACKGROUND The Hybrid emergency room (ER) system is a novel trauma workflow that uses angio-computed tomography (CT) equipment in a trauma resuscitation room. Although the Hybrid ER system decreases time to start surgery and endovascular treatments and improves mortality, the optimal target benefitting from this system remained unclear. We aimed to identify a subset of trauma patients likely to receive the greatest benefits from the Hybrid ER. MATERIAL AND METHODS This retrospective cohort study was conducted in a tertiary hospital in Japan from August 2007 to January 2020. We consecutively included severe adult blunt trauma patients (Injury Severity Score [ISS] ≥16) and divided them into two groups: Conventional group (August 2007-July 2011) and Hybrid ER (August 2011-January 2020) group. We evaluated the association between the Hybrid ER group and 28-day mortality using multivariable logistic regression analysis. The 28-day mortality trend during the study period was evaluated with restricted cubic spline analysis. To evaluate heterogeneity of effects within various patient severities, we evaluated whether the patients' ISS modified the effect of the Hybrid ER on survival. RESULTS Among 1050 trauma patients, the Conventional group comprised 360 and Hybrid ER group 690 patients. ISS and probability of survival (Ps) were not significantly different between the groups. Twenty-eight-day mortality was significantly lower in the Hybrid ER group (Ps-adjusted odds ratio, 0.48; 95% confidence interval, 0.32-0.71; p<0.001). Restricted cubic spline analysis revealed that Ps-adjusted 28-day mortality sharply decreased approximately 200 days after installation of the Hybrid ER. Increase of survival probabilities according to the increase of ISS was significantly improved in Hybrid ER group (p=0.014). As ISS increased to >25, survival probabilities in the Hybrid ER group were higher compared with those in the Conventional group. CONCLUSION The Hybrid ER may improve post-traumatic mortality, especially in patients with higher baseline severity. LEVEL OF EVIDENCE Level-III.
Spatial orientation in virtual environment compared to real-world

Abstract Virtual reality (VR) is popular across many disciplines and has been increasingly used in sports as a training tool lately. However, it is not clear whether the spatial orientation of humans works equally within VR and in the real-world. In this paper, two studies are presented, in which natural body movements were allowed and demanded. Firstly, a series of verbal and walking distance estimation tests were conducted in both the virtual and the real environment. The non-parametric Friedman test with pairwise comparisons showed no significant differences neither in verbal nor in walking distance estimations between the conditions (all p > 0.05). However, shorter distances (0.9-1.5 m) were estimated more precisely than larger distances (2.6-2.8 m) in both environments. Secondly, a self-developed route recall test to examine the spatial orientation was performed in the virtual and the real environment. The participants visually perceived the predefined route and were instructed to follow these routes with their eyes blindfolded and afterward to return to their starting position. Between the ending and the starting position, no difference between the two environments was observed (p > 0.05). Based on these two studies, the performance of the human spatial orientation preliminarily verified the same in a virtual and real environment.
Seismic sensitivity and bone conduction mechanisms enable extratympanic hearing in salamanders

ABSTRACT The tympanic middle ear is an adaptive sensory novelty that evolved multiple times in all the major terrestrial tetrapod groups to overcome the impedance mismatch generated when aerial sound encounters the air–skin boundary. Many extant tetrapod species have lost their tympanic middle ears, yet they retain the ability to detect airborne sound. In the absence of a functional tympanic ear, extratympanic hearing may occur via the resonant qualities of air-filled body cavities, sensitivity to seismic vibration, and/or bone conduction pathways to transmit sound from the environment to the ear. We used auditory brainstem response recording and laser vibrometry to assess the contributions of these extratympanic pathways for airborne sound in atympanic salamanders. We measured auditory sensitivity thresholds in eight species and observed sensitivity to low-frequency sound and vibration from 0.05–1.2 kHz and 0.02–1.2 kHz, respectively. We determined that sensitivity to airborne sound is not facilitated by the vibrational responsiveness of the lungs or mouth cavity. We further observed that, although seismic sensitivity probably contributes to sound detection under naturalistic scenarios, airborne sound stimuli presented under experimental conditions did not produce vibrations detectable to the salamander ear. Instead, threshold-level sound pressure is sufficient to generate translational movements in the salamander head, and these sound-induced head vibrations are detectable by the acoustic sensors of the inner ear. This extratympanic hearing mechanism mediates low-frequency sensitivity in vertebrate ears that are unspecialized for the detection of aerial sound pressure, and may represent a common mechanism for terrestrial hearing across atympanic tetrapods. Summary: Terrestrial hearing in salamanders that lack a tympanic middle ear is mediated by bone conduction of sound that is detected by the acoustic end organs of the inner ear.
Effect of the Nonsteroidal Mineralocorticoid Receptor Blocker, Esaxerenone, on Nocturnal Hypertension: A Post Hoc Analysis of the ESAX-HTN Study

Abstract BACKGROUND Nocturnal hypertension is an important phenotype of abnormal diurnal blood pressure (BP) variability and a known risk marker for target organ damage and cardiovascular events. This study aimed to assess the differential BP-lowering effects of esaxerenone vs. eplerenone on nocturnal BP in hypertensive patients with different nocturnal dipping patterns. METHODS This was a post hoc analysis of the “Esaxerenone (CS-3150) Compared to Eplerenone in Patients with Essential Hypertension” study (NCT02890173), which was a phase 3, multicenter, randomized, controlled, double-blind, parallel-group clinical study conducted in Japan. Ambulatory BP monitoring data were collected. RESULTS Patients (n = 1,001) were randomized to esaxerenone 2.5 mg/day (n = 331) or 5 mg/day (n = 338), or eplerenone 50 mg/day (n = 332). Reductions in nighttime systolic BP (95% confidence interval) were significantly greater with 2.5 and 5 mg/day esaxerenone vs. eplerenone (−2.6 [−5.0, −0.2] and −6.4 mm Hg [−8.8, −4.0], respectively). Esaxerenone significantly reduced nighttime BP from baseline compared with eplerenone in non-dippers with previously uncontrolled BP. In addition, esaxerenone did not markedly alter nighttime BP in extreme dipper patients. In the esaxerenone 5 mg/day group, esaxerenone-induced decreases in nighttime BP were greater than eplerenone-induced decreases in older patients. CONCLUSIONS Esaxerenone may be an effective treatment option for nocturnal hypertension, especially in older patients and those with a non-dipper pattern of nocturnal BP.
Do seasonal microbiome changes affect infection susceptibility, contributing to seasonal disease outbreaks?

The aim of the present paper is to explore whether seasonal outbreaks of infectious diseases may be linked to changes in host microbiomes. This is a very important issue, because one way to have more control over seasonal outbreaks is to understand the factors that underlie them. In this paper, I will evaluate the relevance of the microbiome as one of such factors. The paper is based on two pillars of reasoning. Firstly, on the idea that microbiomes play an important role in their hosts’ defence against infectious diseases. Secondly, on the idea that microbiomes are not stable, but change seasonally. These two ideas are combined in order to argue that seasonal changes in a given microbiome may influence the functionality of the host's immune system and consequently make it easier for infectious agents to infect the host at certain times of year. I will argue that, while this is only a theoretical possibility, certain studies may back up such claims. Furthermore, I will show that this does not necessarily contradict other hypotheses aimed at explaining seasonal outbreaks; in fact, it may even enhance them.