Miracle-dz/UniFAR-FTUs huggingface dataset:
NEREL-BIO: a dataset of biomedical abstracts annotated with nested named entities

Abstract Motivation This article describes NEREL-BIO—an annotation scheme and corpus of PubMed abstracts in Russian and smaller number of abstracts in English. NEREL-BIO extends the general domain dataset NEREL by introducing domain-specific entity types. NEREL-BIO annotation scheme covers both general and biomedical domains making it suitable for domain transfer experiments. NEREL-BIO provides annotation for nested named entities as an extension of the scheme employed for NEREL. Nested named entities may cross entity boundaries to connect to shorter entities nested within longer entities, making them harder to detect. Results NEREL-BIO contains annotations for 700+ Russian and 100+ English abstracts. All English PubMed annotations have corresponding Russian counterparts. Thus, NEREL-BIO comprises the following specific features: annotation of nested named entities, it can be used as a benchmark for cross-domain (NEREL → NEREL-BIO) and cross-language (English → Russian) transfer. We experiment with both transformer-based sequence models and machine reading comprehension models and report their results. Availability and implementation The dataset and annotation guidelines are freely available at https://github.com/nerel-ds/NEREL-BIO.
Analysis of GMRES for Low‐Rank and Small‐Norm Perturbations of the Identity Matrix

In many applications, linear systems arise where the coefficient matrix takes the special form I + K + E, where I is the identity matrix of dimension n, rank(K) = p ≪ n, and ∥E∥ ≤ ϵ < 1. GMRES convergence rates for linear systems with coefficient matrices of the forms I+K and I+E are guaranteed by well‐known theory, but only relatively weak convergence bounds specific to matrices of the form I + K + E currently exist. In this paper, we explore the convergence properties of linear systems with such coefficient matrices by considering the pseudospectrum of I + K. We derive a bound for the GMRES residual in terms of ϵ when approximately solving the linear system (I + K + E)x = b and identify the eigenvalues of I + K that are sensitive to perturbation. In particular, while a clustered spectrum away from the origin is often a good indicator of fast GMRES convergence, that convergence may be slow when some of those eigenvalues are ill‐conditioned. We show there can be at most 2p eigenvalues of I + K that are sensitive to small perturbations. We present numerical results when using GMRES to solve a sequence of linear systems of the form (I + Kj + Ej)xj = bj that arise from the application of Broyden's method to solve a nonlinear partial differential equation.
Adversarial Transformer for Repairing Human Airway Segmentation

Automated airway segmentation models often suffer from discontinuities in peripheral bronchioles, which limits their clinical applicability. Furthermore, data heterogeneity across different centres and pathological abnormalities pose significant challenges to achieving accurate and robust segmentation in distal small airways. Accurate segmentation of airway structures is essential for the diagnosis and prognosis of lung diseases. To address these issues, we propose a patch-scale adversarial-based refinement network that takes in preliminary segmentation and original CT images and outputs a refined mask of the airway structure. Our method is validated on three datasets, including healthy cases, pulmonary fibrosis, and COVID-19 cases, and quantitatively evaluated using seven metrics. Our method achieves more than a 15% increase in the detected length ratio and detected branch ratio compared to previously proposed models, demonstrating its promising performance. The visual results show that our refinement approach, guided by a patch-scale discriminator and centreline objective functions, effectively detects discontinuities and missing bronchioles. We also demonstrate the generalizability of our refinement pipeline on three previous models, significantly improving their segmentation completeness. Our method provides a robust and accurate airway segmentation tool that can help improve diagnosis and treatment planning for lung diseases.
Cox-Hawkes: doubly stochastic spatiotemporal Poisson processes

Hawkes processes are point process models that have been used to capture self-excitatory behavior in social interactions, neural activity, earthquakes and viral epidemics. They can model the occurrence of the times and locations of events. Here we develop a new class of spatiotemporal Hawkes processes that can capture both triggering and clustering behavior and we provide an efficient method for performing inference. We use a log-Gaussian Cox process (LGCP) as prior for the background rate of the Hawkes process which gives arbitrary flexibility to capture a wide range of underlying background effects (for infectious diseases these are called endemic effects). The Hawkes process and LGCP are computationally expensive due to the former having a likelihood with quadratic complexity in the number of observations and the latter involving inversion of the precision matrix which is cubic in observations. Here we propose a novel approach to perform MCMC sampling for our Hawkes process with LGCP background, using pre-trained Gaussian Process generators which provide direct and cheap access to samples during inference. We show the efficacy and flexibility of our approach in experiments on simulated data and use our methods to uncover the trends in a dataset of reported crimes in the US.
Amos: An Adam-style Optimizer with Adaptive Weight Decay towards Model-Oriented Scale

We present Amos, a stochastic gradient-based optimizer designed for training deep neural networks. It can be viewed as an Adam optimizer with theoretically supported, adaptive learning-rate decay and weight decay. A key insight behind Amos is that it leverages model-specific information to determine the initial learning-rate and decaying schedules. When used for pre-training BERT variants and T5, Amos consistently converges faster than the state-of-the-art settings of AdamW, achieving better validation loss within<=70% training steps and time, while requiring<=51% memory for slot variables. Our code is open-sourced at: https://github.com/google-research/jestimator
GLCC: A General Framework for Graph-level Clustering

This paper studies the problem of graph-level clustering, which is a novel yet challenging task. This problem is critical in a variety of real-world applications such as protein clustering and genome analysis in bioinformatics. Recent years have witnessed the success of deep clustering coupled with graph neural networks (GNNs). However, existing methods focus on clustering among nodes given a single graph, while exploring clustering on multiple graphs is still under-explored. In this paper, we propose a general graph-level clustering framework named Graph-Level Contrastive Clustering (GLCC) given multiple graphs. Specifically, GLCC first constructs an adaptive affinity graph to explore instance- and cluster-level contrastive learning (CL). Instance-level CL leverages graph Laplacian based contrastive loss to learn clustering-friendly representations while cluster-level CL captures discriminative cluster representations incorporating neighbor information of each sample. Moreover, we utilize neighbor-aware pseudo-labels to reward the optimization of representation learning. The two steps can be alternatively trained to collaborate and benefit each other. Experiments on a range of well-known datasets demonstrate the superiority of our proposed GLCC over competitive baselines.
Experiencer-Specific Emotion and Appraisal Prediction

Emotion classification in NLP assigns emotions to texts, such as sentences or paragraphs. With texts like “I felt guilty when he cried”, focusing on the sentence level disregards the standpoint of each participant in the situation: the writer (“I”) and the other entity (“he”) could in fact have different affective states. The emotions of different entities have been considered only partially in emotion semantic role labeling, a task that relates semantic roles to emotion cue words. Proposing a related task, we narrow the focus on the experiencers of events, and assign an emotion (if any holds) to each of them. To this end, we represent each emotion both categorically and with appraisal variables, as a psychological access to explaining why a person develops a particular emotion. On an event description corpus, our experiencer-aware models of emotions and appraisals outperform the experiencer-agnostic baselines, showing that disregarding event participants is an oversimplification for the emotion detection task.
RVE Analysis in LS-DYNA for High-fidelity Multiscale Material Modeling

In modern engineering designs, advanced materials (e.g., fiber/particle-reinforced polymers, metallic alloys, laminar composites, etc.) are widely used, where microscale heterogeneities such as grains, inclusions, voids, micro-cracks, and interfaces significantly affect the macroscopic constitutive behaviors. Obviously, an accurate description of the multiscale material behaviors is of great importance to the success of material design and structural analysis. The Representative Volume Element (RVE) analysis method provides a rigorous means to obtain homogenized macroscopic material properties at the upper length scale from the properties of the material constituents and structures at a lower length scale. Recently, we have developed an RVE module (keyword: *RVE_ANALYSIS_FEM) in the multiphysics simulation software LS-DYNA to enable high-fidelity virtual testing of numerically re-constructed material samples at user-specified characteristic length scales. In this article, a brief introduction to this new feature will be given.
Star formation in IC1396. Kinematics and subclusters structure reveled by Gaia

Aims. We investigate the star formation history of the IC1396 region by studying its kinematics and completing the population census. Methods. We use multiwavelength data, combining optical spectroscopy (to identify and classify new members), near-infrared pho- tometry (to trace shocks, jets, and outﬂows and the interactions between the cluster members and the cloud), along with Gaia EDR3 to identify new potential members in the multidimensional proper motion / parallax space. Results. The revised Gaia EDR3 distance is 925 ± 73 pc, slightly closer than previously obtained with DR2. The Gaia data reveal four distinct subclusters in the region. These subclusters are consistent in distance but display di ﬀ erences in proper motion. This, with their age di ﬀ erences, hints towards a complex and varied star formation history. Gaia data also unveil the intermediate-mass objects that tend to evade spectroscopic and disk surveys. Our analysis allows us to identify 334 new members. We estimate an average age of ∼ 4 Myr, conﬁrming previous age estimates. With the new members added to our study, we estimate a disk fraction of 28%, lower than previous values, due to our method detecting mainly new, diskless intermediate-mass stars. We ﬁnd age di ﬀ erences between the subclusters, which evidences a complex star formation history with di ﬀ erent episodes of star formation.
CONSTRAINTS ON BLACK HOLE GROWTH, QUASAR LIFETIMES, AND EDDINGTON RATIO DISTRIBUTIONS FROM THE SDSS BROAD-LINE QUASAR BLACK HOLE MASS FUNCTION

We present an estimate of the black hole mass function of broad-line quasars (BLQSOs) that self-consistently corrects for incompleteness and the statistical uncertainty in the mass estimates, based on a sample of 9886 quasars at 1 < z < 4.5 drawn from the Sloan Digital Sky Survey (SDSS). We find evidence for “cosmic downsizing” of black holes in BLQSOs, where the peak in their number density shifts to higher redshift with increasing black hole mass. The cosmic mass density for black holes seen as BLQSOs peaks at z ∼ 2. We estimate the completeness of the SDSS as a function of the black hole mass and Eddington ratio, and find that at z > 1 it is highly incomplete at MBH ≲ 109 M☉ and L/LEdd ≲ 0.5. We estimate a lower limit on the lifetime of a single BLQSO phase to be tBL > 150 ± 15 Myr for black holes at z = 1 with a mass of MBH = 109 M☉, and we constrain the maximum mass of a black hole in a BLQSO to be ∼3 × 1010 M☉. Our estimated distribution of BLQSO Eddington ratios peaks at L/LEdd ∼ 0.05 and has a dispersion of ∼0.4 dex, implying that most BLQSOs are not radiating at or near the Eddington limit; however, the location of the peak is subject to considerable uncertainty. The steep increase in number density of BLQSOs toward lower Eddington ratios is expected if the BLQSO accretion rate monotonically decays with time. Furthermore, our estimated lifetime and Eddington ratio distributions imply that the majority of the most massive black holes spend a significant amount of time growing in an earlier obscured phase, a conclusion which is independent of the unknown obscured fraction. These results are consistent with models for self-regulated black hole growth, at least for massive systems at z > 1, where the BLQSO phase occurs at the end of a fueling event when black hole feedback unbinds the accreting gas, halting the accretion flow.
gSuite: A Flexible and Framework Independent Benchmark Suite for Graph Neural Network Inference on GPUs

As the interest to Graph Neural Networks (GNNs) is growing, the importance of benchmarking and performance characterization studies of GNNs is increasing. So far, we have seen many studies that investigate and present the performance and computational efficiency of GNNs. However, the work done so far has been carried out using a few high-level GNN frameworks. Although these frameworks provide ease of use, they contain too many dependencies to other existing libraries. The layers of implementation details and the dependencies complicate the performance analysis of GNN models that are built on top of these frameworks, especially while using architectural simulators. Furthermore, different approaches on GNN computation are generally overlooked in prior characterization studies, and merely one of the common computational models is evaluated. Based on these shortcomings and needs that we observed, we developed a benchmark suite that is framework independent, supporting versatile computational models, easily configurable and can be used with architectural simulators without additional effort.Our benchmark suite, which we call gSuite, makes use of only hardware vendor’s libraries and therefore it is independent of any other frameworks. gSuite enables performing detailed performance characterization studies on GNN Inference using both contemporary GPU profilers and architectural GPU simulators. To illustrate the benefits of our new benchmark suite, we perform a detailed characterization study with a set of well-known GNN models with various datasets; running gSuite both on a real GPU card and a timing-detailed GPU simulator. We also implicate the effect of computational models on performance. We use several evaluation metrics to rigonously measure the performance of GNN computation. We make gSuite available to research community and provide all the configuration settings which we used for our evaluation so that all the experiments mentioned in the paper are reproducible.
Energy stable schemes for gradient flows based on the DVD method

The existing discrete variational derivative method is only second-order accurate and fully implicit. In this paper, we propose a framework to construct an arbitrary high-order implicit (original) energy stable scheme and a second-order semi-implicit (modified) energy stable scheme. Combined with the Runge--Kutta process, we can build an arbitrary high-order and unconditionally (original) energy stable scheme based on the discrete variational derivative method. The new energy stable scheme is implicit and leads to a large sparse nonlinear algebraic system at each time step, which can be efficiently solved by using an inexact Newton type algorithm. To avoid solving nonlinear algebraic systems, we then present a relaxed discrete variational derivative method, which can construct second-order, linear, and unconditionally (modified) energy stable schemes. Several numerical simulations are performed to investigate the efficiency, stability, and accuracy of the newly proposed schemes.
m4Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter

Multilingual neural machine translation models (MNMT) yield state-of-the-art performance when evaluated on data from a domain and language pair seen at training time. However, when a MNMT model is used to translate under domain shift or to a new language pair, performance drops dramatically. We consider a very challenging scenario: adapting the MNMT model both to a new domain and to a new language pair at the same time. In this paper, we propose $m^4Adapter$ (Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter), which combines domain and language knowledge using meta-learning with adapters. We present results showing that our approach is a parameter-efficient solution which effectively adapts a model to both a new language pair and a new domain, while outperforming other adapter methods. An ablation study also shows that our approach more effectively transfers domain knowledge across different languages and language information across different domains.
ANALYSIS OF THE GRADIENT FOR THE STOCHASTIC FRACTIONAL HEAT EQUATION WITH SPATIALLY-COLORED NOISE IN R d

: Consider the stochastic partial diﬀerential equation where ´p´ ∆ q α 2 denotes the fractional Laplacian with the power α { 2 P p 1 { 2 , 1 s , and the driving noise 9 F is a centered Gaussian ﬁeld which is white in time and with a spatial homogeneous covariance given by the Riesz kernel. We study the detailed behavior of the approximation spatial gradient u t p x q ´ u t p x ´ ε e q at any ﬁxed time t ą 0, as ε Ó 0, where e is the unit vector in R d . As applications, we deduce the law of iterated logarithm and the behavior of the q -variations of the solution in space. 60H15;
Hybrid approach analysis of energy detection and eigenvalue based spectrum sensing algorithms with noise power estimation

Two particular semi-blind spectrum sensing algorithms are taken into account in this paper: Energy Detection (ED) and Roy's Largest Root Test (RLRT). Both algorithms require the knowledge of the noise power in order to achieve optimal performance. Since by its nature the noise power is unpredictable, noise variance estimation is needed in order to cope with the absence of prior knowledge of the noise power: this leads to a new hybrid approach for both considered detectors. Probability of detection and false alarm with this new approach are derived in closed-form expressions. The impact of noise estimation accuracy for ED and RLRT is evaluated in terms of Receiver Operating Characteristic (ROC) curves and performance curves, i.e., detection/misdetection probability as a function of the Signal to Noise Ratio (SNR). Analytical results have been confirmed by numerical simulations under a flat-fading channel scenario. It is concluded that both hybrid approaches tend to their ideal cases when a large number of slots is used for noise variance estimation and that the impairment due to noise uncertainty is reduced on RLRT w.r. t. ED
Transferring learned patterns from ground-based field imagery to predict UAV-based imagery for crop and weed semantic segmentation in precision crop farming

: Weed and crop segmentation is becoming an increasingly integral part of precision farming that leverages the current computer vision and deep learning technologies. Research has been extensively carried out based on images captured with a camera from various platforms. Unmanned aerial vehicles (UAVs) and ground-based vehicles including agricultural robots are the two popular platforms for data collection in fields. They all contribute to site-specific weed management (SSWM) to maintain crop yield. Currently, the data from these two platforms is processed separately, though sharing the same semantic objects (weed and crop). In our paper, we have developed a deep convolutional network that enables to predict both field and aerial images from UAVs for weed segmentation and mapping with only field images provided in the training phase. The network learning process is visualized by feature maps at shallow and deep layers. The results show that the mean intersection of union (IOU) values of the segmentation for the crop (maize), weeds, and soil background in the developed model for the field dataset are 0.744, 0.577, 0.979, respectively, and the performance of aerial images from an UAV with the same model, the IOU values of the segmentation for the crop (maize), weeds and soil background are 0.596, 0.407, and 0.875, respectively. To estimate the effect on the use of plant protection agents, we quantify the relationship between herbicide spraying saving rate and grid size (spraying resolution) based on the predicted weed map. The spraying saving rate is up to 90% when the spraying resolution is at 1.78×1.78 cm 2 . The study shows that the developed deep convolutional neural network could be used to classify weeds from both field and aerial images and delivers satisfactory results.
5-LIST-COLORING GRAPHS ON SURFACES

setting. In Section 5.8, we apply the general theory to the family of 6-listcritical graphs to derive the main results for 5-list-coloring. Finally, in Section 5.9, we apply the theory for a slightly different family to obtain the exponentially many 5-list-colorings result. 5.2 A Linear Bound for the Cylinder Theorem 5.2. 1. [Cylinder Theorem: Cycles with Lists of Size Three] If Γ = (G,C ∪ C ′, L) is a cylinder-canvas where vertices in C ∪ C ′ have lists of size at least three and d(C,C ′) ≥ D, then there exists an L-coloring of G. Proof. Suppose that Γ is a minimum counterexample to the theorem. Thus Γ is critical. Hence there are no degree four vertices in G\ (C ∪C ′) and there do not exist vertices in the interior of triangles that do not separate a vertex of C from a vertex
Tag-Set-Sequence Learning for Generating Question-Answer Pairs

: Transformer-based QG models can generate question-answer pairs (QAPs) with high qualities, but may also generate silly questions for certain texts. We present a new method called tag-set sequence learning to tackle this problem, where a tag-set sequence is a sequence of tag sets to capture the syntactic and semantic information of the underlying sentence, and a tag set consists of one or more language feature tags, including, for example, semantic-role-labeling, part-of-speech, named-entity-recognition, and sentiment-indication tags. We construct a system called TSS-Learner to learn tag-set sequences from given declarative sentences and the corresponding interrogative sentences, and derive answers to the latter. We train a TSS-Learner model for the English language using a small training dataset and show that it can indeed generate adequate QAPs for certain texts that transformer-based models do poorly. Human evaluation on the QAPs generated by TSS-Learner over SAT practice reading tests is encouraging.
Characterization of Multi-Link Propagation and Bistatic Target Reﬂectivity for Distributed ISAC

—Integrated sensing and communication (ISAC) qual- iﬁes mobile radio systems for detecting and localizing of passive objects by means of radar sensing. Advanced ISAC networks rely on meshed mobile radio access nodes (infrastructure and/or user equipment, resp.) establishing a distributed, multistatic MIMO radar system in which each target reveals itself by its bistatic backscattering. Therefore, characterization of the bistatic reﬂectivity of targets along their trajectories of movement is of highest importance for ISAC performance prediction. We sum- marize several challenges in bistatic modeling and measurement of extended, potentially time-variant radar targets. We emphasize the speciﬁc challenges arising for distributed (hence multi-link) ISAC networks and compare to the state of the art in propagation modeling for mobile communication.