Miracle-dz/UniFAR-FTUs huggingface dataset:
Synchronous Dual Network with Cross-Type Attention for Joint Entity and Relation Extraction

Joint entity and relation extraction is challenging due to the complex interaction of interaction between named entity recognition and relation extraction. Although most existing works tend to jointly train these two tasks through a shared network, they fail to fully utilize the interdependence between entity types and relation types. In this paper, we design a novel synchronous dual network (SDN) with cross-type attention via separately and interactively considering the entity types and relation types. On the one hand, SDN adopts two isomorphic bi-directional type-attention LSTM to encode the entity type enhanced representations and the relation type enhanced representations, respectively. On the other hand, SDN explicitly models the interdependence between entity types and relation types via cross-type attention mechanism. In addition, we also propose a new multi-task learning strategy via modeling the interaction of two types of information. Experiments on NYT and WebNLG datasets verify the effectiveness of the proposed model, achieving state-of-the-art performance.
Fine-grained Entity Typing without Knowledge Base

Existing work on Fine-grained Entity Typing (FET) typically trains automatic models on the datasets obtained by using Knowledge Bases (KB) as distant supervision. However, the reliance on KB means this training setting can be hampered by the lack of or the incompleteness of the KB. To alleviate this limitation, we propose a novel setting for training FET models: FET without accessing any knowledge base. Under this setting, we propose a two-step framework to train FET models. In the first step, we automatically create pseudo data with fine-grained labels from a large unlabeled dataset. Then a neural network model is trained based on the pseudo data, either in an unsupervised way or using self-training under the weak guidance from a coarse-grained Named Entity Recognition (NER) model. Experimental results show that our method achieves competitive performance with respect to the models trained on the original KB-supervised datasets.
Sociolectal Analysis of Pretrained Language Models

Using data from English cloze tests, in which subjects also self-reported their gender, age, education, and race, we examine performance differences of pretrained language models across demographic groups, defined by these (protected) attributes. We demonstrate wide performance gaps across demographic groups and show that pretrained language models systematically disfavor young non-white male speakers; i.e., not only do pretrained language models learn social biases (stereotypical associations) – pretrained language models also learn sociolectal biases, learning to speak more like some than like others. We show, however, that, with the exception of BERT models, larger pretrained language models reduce some the performance gaps between majority and minority groups.
ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations

Understanding how events are semantically related to each other is the essence of reading comprehension. Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. While these tasks partially evaluate machines’ ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning. For example, to understand causality between events, we need to infer motivation or purpose; to establish event hierarchy, we need to understand the composition of events. To facilitate these tasks, we introduce **ESTER**, a comprehensive machine reading comprehension (MRC) dataset for Event Semantic Relation Reasoning. The dataset leverages natural language queries to reason about the five most common event semantic relations, provides more than 6K questions, and captures 10.1K event relation pairs. Experimental results show that the current SOTA systems achieve 22.1%, 63.3% and 83.5% for token-based exact-match (**EM**), **F1** and event-based **HIT@1** scores, which are all significantly below human performances (36.0%, 79.6%, 100% respectively), highlighting our dataset as a challenging benchmark.
Benchmarking Deep Learning Models on Point Cloud Segmentation

Point clouds are currently used for a variety of applications, such as detection tasks in medical and geological domains. Intelligent analysis of point clouds is considered a highly computationally demanding and challenging task, especially the segmentation task among the points. Although numerous deep learning models have recently been proposed to segment point cloud data, there is no clear instruction of which exactly neural network to utilize and then incorporate into a system dealing with point cloud segmentation analysis. Besides, the majority of the developed models emphasize more on the accuracy rather than the efficiency, in order to achieve great results. Consequently, the training, validation and testing phases of the models require a great number of processing hours and a huge amount of memory. These high computational requirements are commonly difficult to deal with for many users. In this article, we analyse five state-of-the-art deep learning models for part segmentation task and give meaningful insights into the utilization of each one. We advance guidelines based on different properties, considering both learning-related metrics, such as accuracy, and system-related metrics, such as run time and memory footprint. We further propose and analyse generalized performance metrics, which facilitate the model evaluation phase in segmentation tasks allowing users to select the most appropriate approach for their context in terms of accuracy and efficiency.
Generation and Extraction Combined Dialogue State Tracking with Hierarchical Ontology Integration

Recently, the focus of dialogue state tracking has expanded from single domain to multiple domains. The task is characterized by the shared slots between domains. As the scenario gets more complex, the out-of-vocabulary problem also becomes severer. Current models are not satisfactory for solving the challenges of ontology integration between domains and out-of-vocabulary problems. To address the problem, we explore the hierarchical semantic of ontology and enhance the interrelation between slots with masked hierarchical attention. In state value decoding stage, we solve the out-of-vocabulary problem by combining generation method and extraction method together. We evaluate the performance of our model on two representative datasets, MultiWOZ in English and CrossWOZ in Chinese. The results show that our model yields a significant performance gain over current state-of-the-art state tracking model and it is more robust to out-of-vocabulary problem compared with other methods.
Time-aware Graph Neural Network for Entity Alignment between Temporal Knowledge Graphs

Entity alignment aims to identify equivalent entity pairs between different knowledge graphs (KGs). Recently, the availability of temporal KGs (TKGs) that contain time information created the need for reasoning over time in such TKGs. Existing embedding-based entity alignment approaches disregard time information that commonly exists in many large-scale KGs, leaving much room for improvement. In this paper, we focus on the task of aligning entity pairs between TKGs and propose a novel Time-aware Entity Alignment approach based on Graph Neural Networks (TEA-GNN). We embed entities, relations and timestamps of different KGs into a vector space and use GNNs to learn entity representations. To incorporate both relation and time information into the GNN structure of our model, we use a self-attention mechanism which assigns different weights to different nodes with orthogonal transformation matrices computed from embeddings of the relevant relations and timestamps in a neighborhood. Experimental results on multiple real-world TKG datasets show that our method significantly outperforms the state-of-the-art methods due to the inclusion of time information.
Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport

Timeline Summarization identifies major events from a news collection and describes them following temporal order, with key dates tagged. Previous methods generally generate summaries separately for each date after they determine the key dates of events. These methods overlook the events’ intra-structures (arguments) and inter-structures (event-event connections). Following a different route, we propose to represent the news articles as an event-graph, thus the summarization becomes compressing the whole graph to its salient sub-graph. The key hypothesis is that the events connected through shared arguments and temporal order depict the skeleton of a timeline, containing events that are semantically related, temporally coherent and structurally salient in the global event graph. A time-aware optimal transport distance is then introduced for learning the compression model in an unsupervised manner. We show that our approach significantly improves on the state of the art on three real-world datasets, including two public standard benchmarks and our newly collected Timeline100 dataset.
Effects of Seagrass Wasting Disease on Eelgrass Growth and Belowground Sugar in Natural Meadows

Seagrass meadows provide valuable ecosystem benefits but are at risk from disease. Eelgrass (Zostera marina) is a temperate species threatened by seagrass wasting disease (SWD), caused by the protist Labyrinthula zosterae. The pathogen is sensitive to warming ocean temperatures, prompting a need for greater understanding of the impacts on host health under climate change. Previous work demonstrates pathogen cultures grow faster under warmer laboratory conditions and documents positive correlations between warmer ocean temperatures and disease levels in nature. However, the consequences of disease outbreaks on eelgrass growth remain poorly understood. Here, we examined the effect of disease on eelgrass productivity in the field. We coupled in situ shoot marking with high-resolution imagery of eelgrass blades and used an artificial intelligence application to determine disease prevalence and severity from digital images. Comparisons of eelgrass growth and disease metrics showed that SWD impaired eelgrass growth and accumulation of non-structural carbon in the field. Blades with more severe disease had reduced growth rates, indicating that disease severity can limit plant growth. Disease severity and rhizome sugar content were also inversely related, suggesting that disease reduced belowground carbon accumulation. Finally, repeated measurements of diseased blades indicated that lesions can grow faster than healthy tissue in situ. This is the first study to demonstrate the negative impact of wasting disease on eelgrass health in a natural meadow. These results emphasize the importance of considering disease alongside other stressors to better predict the health and functioning of seagrass meadows in the Anthropocene.
Evaluating Cancer-Related Biomarkers Based on Pathological Images: A Systematic Review

Many diseases are accompanied by changes in certain biochemical indicators called biomarkers in cells or tissues. A variety of biomarkers, including proteins, nucleic acids, antibodies, and peptides, have been identified. Tumor biomarkers have been widely used in cancer risk assessment, early screening, diagnosis, prognosis, treatment, and progression monitoring. For example, the number of circulating tumor cell (CTC) is a prognostic indicator of breast cancer overall survival, and tumor mutation burden (TMB) can be used to predict the efficacy of immune checkpoint inhibitors. Currently, clinical methods such as polymerase chain reaction (PCR) and next generation sequencing (NGS) are mainly adopted to evaluate these biomarkers, which are time-consuming and expansive. Pathological image analysis is an essential tool in medical research, disease diagnosis and treatment, functioning by extracting important physiological and pathological information or knowledge from medical images. Recently, deep learning-based analysis on pathological images and morphology to predict tumor biomarkers has attracted great attention from both medical image and machine learning communities, as this combination not only reduces the burden on pathologists but also saves high costs and time. Therefore, it is necessary to summarize the current process of processing pathological images and key steps and methods used in each process, including: (1) pre-processing of pathological images, (2) image segmentation, (3) feature extraction, and (4) feature model construction. This will help people choose better and more appropriate medical image processing methods when predicting tumor biomarkers.
Subdistrict-Level Reproductive Number for Foot and Mouth Disease in Cattle in Northern Thailand

Foot and mouth disease (FMD) is an important contagious transboundary disease that causes a significant economic loss for several countries. The FMD virus (FMDV) can spread very rapidly by direct and indirect transmission among susceptible animals. The complexity and magnitude of FMDV transmission at the initial stages of the epidemic can be expressed by the basic reproductive number (R0), and furthermore, control strategies can be assessed by the estimation of the effective reproductive number. In this study, we aimed to describe FMD outbreaks among smallholder cattle farms by subdistricts in the northern Thailand and compute the effective reproductive number for outbreaks caused by FMDV serotype O and overall serotypes, including serotype O, serotype A, and unidentified serotype, at the subdistrict level (Rsd) using an epidemic doubling time method. Field data of FMD outbreaks during 2015–2017 that affected 94 subdistricts in northern Thailand were assessed to estimate the Rsd. Results showed that 63.38% (90/142) of the FMD outbreak episodes in cattle were caused by FMDV serotype O. The average doubling time and the Rsd estimated of the outbreaks caused by FMDV serotype O and overall serotype were 2.80 and 4.67 months, and 1.06 and 1.04, respectively. Our results indicated that transmission of FMD in cattle at the subdistrict level in northern Thailand was not controlled (Rsd > 1), which indicates the endemicity of the disease in the region. Although control measures are in place, the results from this study highlighted the need for enhancing FMD monitoring and control strategies in northern Thailand.
Large Multicohort Study Reveals a Prostate Cancer Susceptibility Allele at 5p15 Regulating TERT via Androgen Signaling-Orchestrated Chromatin Binding of E2F1 and MYC

Aberrant telomerase reverse transcriptase (TERT) expression is crucial for tumor survival and cancer cells escaping apoptosis. Multiple TERT-locus variants at 5p15 have been discovered in association with cancer risk, yet the underlying mechanisms and clinical impacts remain unclear. Here, our association studies showed that the TERT promoter variant rs2853669 confers a risk of prostate cancer (PCa) in different ethnic groups. Further functional investigation revealed that the allele-specific binding of MYC and E2F1 at TERT promoter variant rs2853669 associates with elevated level of TERT in PCa. Mechanistically, androgen stimulations promoted the binding of MYC to allele T of rs2853669, thereby activating TERT, whereas hormone deprivations enhanced E2F1 binding at allele C of rs2853669, thus upregulating TERT expression. Notably, E2F1 could cooperate with AR signaling to regulate MYC expression. Clinical data demonstrated synergistic effects of MYC/E2F1/TERT expression or with the TT and CC genotype of rs2853669 on PCa prognosis and severity. Strikingly, single-nucleotide editing assays showed that the CC genotype of rs2853669 obviously promotes epithelial–mesenchymal transition (EMT) and the development of castration-resistant PCa (CRPC), confirmed by unbiased global transcriptome profiling. Our findings thus provided compelling evidence for understanding the roles of noncoding variations coordinated with androgen signaling and oncogenic transcription factors in mis-regulating TERT expression and driving PCa.
Predicting Bone Metastasis Using Gene Expression-Based Machine Learning Models

Bone is the most common site of distant metastasis from malignant tumors, with the highest prevalence observed in breast and prostate cancers. Such bone metastases (BM) cause many painful skeletal-related events, such as severe bone pain, pathological fractures, spinal cord compression, and hypercalcemia, with adverse effects on life quality. Many bone-targeting agents developed based on the current understanding of BM onset’s molecular mechanisms dull these adverse effects. However, only a few studies investigated potential predictors of high risk for developing BM, despite such knowledge being critical for early interventions to prevent or delay BM. This work proposes a computational network-based pipeline that incorporates a ML/DL component to predict BM development. Based on the proposed pipeline we constructed several machine learning models. The deep neural network (DNN) model exhibited the highest prediction accuracy (AUC of 92.11%) using the top 34 featured genes ranked by betweenness centrality scores. We further used an entirely separate, “external” TCGA dataset to evaluate the robustness of this DNN model and achieved sensitivity of 85%, specificity of 80%, positive predictive value of 78.10%, negative predictive value of 80%, and AUC of 85.78%. The result shows the models’ way of learning allowed it to zoom in on the featured genes that provide the added benefit of the model displaying generic capabilities, that is, to predict BM for samples from different primary sites. Furthermore, existing experimental evidence provides confidence that about 50% of the 34 hub genes have BM-related functionality, which suggests that these common genetic markers provide vital insight about BM drivers. These findings may prompt the transformation of such a method into an artificial intelligence (AI) diagnostic tool and direct us towards mechanisms that underlie metastasis to bone events.
Dual Targeting of Multiple Myeloma Stem Cells and Myeloid-Derived Suppressor Cells for Treatment of Chemotherapy-Resistant Multiple Myeloma

Here we review the insights and lessons learned from early clinical trials of T-cell engaging bispecific antibodies (BsABs) as a new class of biotherapeutic drug candidates with clinical impact potential for the treatment of multiple myeloma (MM). BsABs are capable of redirecting host T-cell cytotoxicity in an MHC-independent manner to malignant MM clones as well as immunosuppressive myeloid-derived suppressor cells (MDSC). T-cell engaging BsAB targeting the BCMA antigen may help delay disease progression in MM by destroying the MM cells. T-cell engaging BsAB targeting the CD38 antigen may help delay disease progression in MM by depleting both the malignant MM clones and the MDSC in the bone marrow microenvironment (BMME). BsABs may facilitate the development of a new therapeutic paradigm for achieving improved survival in MM by altering the immunosuppressive BMME. T-cell engaging BsiABs targeting the CD123 antigen may help delay disease progression in MM by depleting the MDSC in the BMME and destroying the MM stem cells that also carry the CD123 antigen on their surface.
MIA SH3 Domain ER Export Factor 3 Deficiency Prevents Neointimal Formation by Restoring BAT-Like PVAT and Decreasing VSMC Proliferation and Migration

Abnormal proliferation and migration of vascular smooth muscle cells (VSMCs) and excessive accumulation of dysfunctional PVAT are hallmarks of pathogenesis after angioplasty. Recent genome-wide association studies reveal that single-nucleotide polymorphism (SNP) in MIA3 is associated with atherosclerosis-relevant VSMC phenotypes. However, the role of MIA3 in the vascular remodeling response to injury remains unknown. Here, we found that expression of MIA3 is increased in proliferative VSMCs and knockdown of MIA3 reduces VSMCs proliferation, migration, and inflammation, whereas MIA3 overexpression promoted VSMC migration and proliferation. Moreover, knockdown of MIA3 ameliorates femoral artery wire injury-induced neointimal hyperplasia and increases brown-like perivascular adipocytes. Collectively, the data suggest that MIA3 deficiency prevents neointimal formation by decreasing VSMC proliferation, migration, and inflammation and maintaining BAT-like perivascular adipocytes in PVAT during injury-induced vascular remodeling, which provide a potential therapeutic target for preventing neointimal hyperplasia in proliferative vascular diseases.
Bioactive Natural Products in Actinobacteria Isolated in Rainwater From Storm Clouds Transported by Western Winds in Spain

Actinobacteria are the main producers of bioactive natural products essential for human health. Although their diversity in the atmosphere remains largely unexplored, using a multidisciplinary approach, we studied here 27 antibiotic producing Actinobacteria strains, isolated from 13 different precipitation events at three locations in Northern and Southern Spain. Rain samples were collected throughout 2013–2016, from events with prevailing Western winds. NOAA HYSPLIT meteorological analyses were used to estimate the sources and trajectories of the air-mass that caused the rainfall events. Five-day backward air masses trajectories of the diverse events reveals a main oceanic source from the North Atlantic Ocean, and in some events long range transport from the Pacific and the Arctic Oceans; terrestrial sources from continental North America and Western Europe were also estimated. Different strains were isolated depending on the precipitation event and the latitude of the sampling site. Taxonomic identification by 16S rRNA sequencing and phylogenetic analysis revealed these strains to belong to two Actinobacteria genera. Most of the isolates belong to the genus Streptomyces, thus increasing the number of species of this genus isolated from the atmosphere. Furthermore, five strains belonging to the rare Actinobacterial genus Nocardiopsis were isolated in some events. These results reinforce our previous Streptomyces atmospheric dispersion model, which we extend herein to the genus Nocardiopsis. Production of bioactive secondary metabolites was analyzed by LC-UV-MS. Comparative analyses of Streptomyces and Nocardiopsis metabolites with natural product databases led to the identification of multiple, chemically diverse, compounds. Among bioactive natural products identified 55% are antibiotics, both antibacterial and antifungal, and 23% have antitumor or cytotoxic properties; also compounds with antiparasitic, anti-inflammatory, immunosuppressive, antiviral, insecticidal, neuroprotective, anti-arthritic activities were found. Our findings suggest that over time, through samples collected from different precipitation events, and space, in different sampling places, we can have access to a great diversity of Actinobacteria producing an extraordinary reservoir of bioactive natural products, from remote and very distant origins, thus highlighting the atmosphere as a contrasted source for the discovery of novel compounds of relevance in medicine and biotechnology.