Miracle-dz/UniFAR-FTUs huggingface dataset:
Dependency-based Mixture Language Models

Various models have been proposed to incorporate knowledge of syntactic structures into neural language models. However, previous works have relied heavily on elaborate components for a specific language model, usually recurrent neural network (RNN), which makes themselves unwieldy in practice to fit into other neural language models, such as Transformer and GPT-2. In this paper, we introduce the Dependency-based Mixture Language Models. In detail, we first train neural language models with a novel dependency modeling objective to learn the probability distribution of future dependent tokens given context. We then formulate the next-token probability by mixing the previous dependency modeling probability distributions with self-attention. Extensive experiments and human evaluations show that our method can be easily and effectively applied to different neural language models while improving neural text generation on various tasks.
BIOS: An Algorithmically Generated Biomedical Knowledge Graph

Biomedical knowledge graphs (BioMedKGs) are essential infrastructures for biomedical and healthcare big data and artificial intelligence (AI), facilitating natural language processing, model development, and data exchange. For decades, these knowledge graphs have been developed via expert curation; however, this method can no longer keep up with today's AI development, and a transition to algorithmically generated BioMedKGs is necessary. In this work, we introduce the Biomedical Informatics Ontology System (BIOS), the first large-scale publicly available BioMedKG generated completely by machine learning algorithms. BIOS currently contains 4.1 million concepts, 7.4 million terms in two languages, and 7.3 million relation triplets. We present the methodology for developing BIOS, including the curation of raw biomedical terms, computational identification of synonymous terms and aggregation of these terms to create concept nodes, semantic type classification of the concepts, relation identification, and biomedical machine translation. We provide statistics on the current BIOS content and perform preliminary assessments of term quality, synonym grouping, and relation extraction. The results suggest that machine learning-based BioMedKG development is a viable alternative to traditional expert curation.
Subreddit Links Drive Community Creation and User Engagement on Reddit

On Reddit, individual subreddits are used to organize content and connect users. One mode of interaction is the subreddit link, which occurs when a user makes a direct reference to a subreddit in another community. Based on the ubiquity of these references, we have undertaken a study on subreddit links on Reddit, with the goal of understanding their impact on both the referenced subreddit, and on the subreddit landscape as a whole. By way of an extensive observational study along with several natural experiments using the entire history of Reddit, we were able to determine that (1) subreddit links are a significant driver of new subreddit creation; (2) subreddit links (2a) substantially drive activity in the referenced subreddit, and (2b) are frequently created in response to high levels of activity in the referenced subreddit; and (3) the graph of subreddit links has become less dense and more treelike over time. We conclude with a discussion of how these results confirm, add to, and in some cases conflict with existing theories on information-seeking behavior and self-organizing behavior in online social systems.
Fourier Disentangled Space-Time Attention for Aerial Video Recognition

We present an algorithm, Fourier Activity Recognition (FAR), for UAV video activity recognition. Our formulation uses a novel Fourier object disentanglement method to innately separate out the human agent (which is typically small) from the background. Our disentanglement technique operates in the frequency domain to characterize the extent of temporal change of spatial pixels, and exploits convolution-multiplication properties of Fourier transform to map this representation to the corresponding object-background entangled features obtained from the network. To encapsulate contextual information and long-range space-time dependencies, we present a novel Fourier Attention algorithm, which emulates the benefits of self-attention by modeling the weighted outer product in the frequency domain. Our Fourier attention formulation uses much fewer computations than self-attention. We have evaluated our approach on multiple UAV datasets including UAV Human RGB, UAV Human Night, Drone Action, and NEC Drone. We demonstrate a relative improvement of 8.02% - 38.69% in top-1 accuracy and up to 3 times faster over prior works.
Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defense

We proposes a novel algorithm, ANTHRO, that inductively extracts over 600K human-written text perturbations in the wild and leverages them for realistic adversarial attack. Unlike existing character-based attacks which often deductively hypothesize a set of manipulation strategies, our work is grounded on actual observations from real-world texts. We find that adversarial texts generated by ANTHRO achieve the best trade-off between (1) attack success rate, (2) semantic preservation of the original text, and (3) stealthiness–i.e. indistinguishable from human writings hence harder to be flagged as suspicious. Specifically, our attacks accomplished around 83% and 91% attack success rates on BERT and RoBERTa, respectively. Moreover, it outperformed the TextBugger baseline with an increase of 50% and 40% in terms of semantic preservation and stealthiness when evaluated by both layperson and professional human workers. ANTHRO can further enhance a BERT classifier’s performance in understanding different variations of human-written toxic texts via adversarial training when compared to the Perspective API.
Enabling matter power spectrum emulation in beyond-ΛCDM cosmologies with COLA

We compare and validate COLA (COmoving Lagrangian Acceleration) simulations against existing emulators in the literature, namely Bacco and Euclid Emulator 2. Our analysis focuses on the non-linear response function, i.e., the ratio between the non-linear dark matter power spectrum in a given cosmology with respect to a pre-defined reference cosmology, which is chosen to be the Euclid Emulator 2 reference cosmology in this paper. We vary three cosmological parameters, the total matter density, the amplitude of the primordial scalar perturbations and the spectral index. By comparing the COLA non-linear response function with those computed from each emulator in the redshift range 0 ≤ z ≤ 3, we find that the COLA method is in excellent agreement with the two emulators for scales up to k ∼ 1 h/Mpc as long as the deviations of the matter power spectrum from the reference cosmology are not too large. We validate the implementation of massive neutrinos in our COLA simulations by varying the sum of neutrino masses to three different values, 0.0 eV, 0.058 eV and 0.15 eV. We show that all three non-linear prescriptions used in this work agree at the 1% level at k ≤ 1 h/Mpc. We then introduce the Effective Field Theory of Dark Energy in our COLA simulations using the N-body gauge method. We consider two different modified gravity models in which the growth of structure is enhanced or suppressed at small scales, and show that the response function with respect to the change of modified gravity parameters depends weakly on cosmological parameters in these models.
Constitutive model characterization and discovery using physics-informed deep learning

Classically, the mechanical response of materials is described through constitutive models, often in the form of constrained ordinary differential equations. These models have a very limited number of parameters, yet, they are extremely efficient in reproducing complex responses observed in experiments. Additionally, in their discretized form, they are computationally very efficient, often resulting in a simple algebraic relation, and therefore they have been extensively used within large-scale explicit and implicit finite element models. However, it is very challenging to formulate new constitutive models, particularly for materials with complex microstructures such as composites. A recent trend in constitutive modeling leverages complex neural network architectures to construct complex material responses where a constitutive model does not yet exist. Whilst very accurate, they suffer from two deficiencies. First, they are interpolation models and often do poorly in extrapolation. Second, due to their complex architecture and numerous parameters, they are inefficient to be used as a constitutive model within a large-scale finite element model. In this study, we propose a novel approach based on the physics-informed learning machines for the characterization and discovery of constitutive models. Unlike data-driven constitutive models, we leverage foundations of elastoplasticity theory as regularization terms in the total loss function to find parametric constitutive models that are also theoretically sound. We demonstrate that our proposed framework can efficiently identify the underlying constitutive model describing different datasets from the von Mises family.
Modeling of a Quadruped Robot with Spine Joints and Full-Dynamics Simulation Environment Construction

This paper presents modeling and simulation of a spined quadruped robot. Extended literature survey is employed and spine joints researches of the quadruped robots are classified. Most of the researchers execute simplified quadruped robot models in their simulations. This survey reveals the need for the full-body spined quadruped simulation environment. First, the kinematics and dynamics modeling of the active spined quadruped robot is obtained. Since quadruped robots are floating-base robots, all derivations are performed with respect to an inertial frame. The motion equations are acquired by the Lagrangian approach. The simulation environment is constructed in the MATLAB/Simulink platform, considering its rich library, powerful solvers, and suitable and resilient environment in integrating controllers. The computation speed of the simulation environment is increased by using optimized MATLAB functions. Precise and accurate contact model is utilized in the simulation environment. We foreseen that the provided full-dynamics simulation environment will be helpful for further spine joint studies on the quadruped robot field.
Consonant-Vowel Transition Models Based on Deep Learning for Objective Evaluation of Articulation

Spectro-temporal dynamics of consonant-vowel (CV) transition regions are considered to provide robust cues related to articulation. In this work, we propose an objective measure of precise articulation, dubbed the objective articulation measure (OAM), by analyzing the CV transitions segmented around vowel onsets. The OAM is derived based on the posteriors of a convolutional neural network pre-trained to classify between different consonants using CV regions as input. We demonstrate that the OAM is correlated with perceptual measures in a variety of contexts including (a) adult dysarthric speech, (b) the speech of children with cleft lip/palate, and (c) a database of accented English speech from native Mandarin and Spanish speakers.
Efficient Algorithms for Extreme Bandits

In this paper, we contribute to the Extreme Bandit problem, a variant of Multi-Armed Bandits in which the learner seeks to collect the largest possible reward. We first study the concentration of the maximum of i.i. d random variables under mild assumptions on the tail of the rewards distributions. This analysis motivates the introduction of Quantile of Maxima (QoMax). The properties of QoMax are sufficient to build an Explore-Then-Commit (ETC) strategy, QoMax-ETC, achieving strong asymptotic guarantees despite its simplicity. We then propose and analyze a more adaptive, anytime algorithm, QoMax-SDA, which combines QoMax with a subsampling method recently introduced by Baudry et al. (2021). Both algorithms are more efficient than existing approaches in two aspects (1) they lead to better empirical performance (2) they enjoy a significant reduction of the memory and time complexities.
Thermal Tides in the Martian Atmosphere Near Northern Summer Solstice Observed by ACS/TIRVIM Onboard TGO

Thermal tides in the Martian atmosphere are analyzed using temperature profiles retrieved from nadir observations obtained by the TIRVIM Fourier‐spectrometer, part of the Atmospheric Chemistry Suite onboard the ExoMars Trace Gas Orbiter. The data is selected near the northern summer solstice at solar longitude (LS) 75°–105° of Martian Year 35. The observations have a full local time coverage, which enables analyses of daily temperature anomalies. The observed zonal mean temperature is lower by 4–6 K at ∼100 Pa, but higher toward the summer pole, compared to the Laboratoire de Météorologie Dynamique (LMD) Mars General Circulation Model (GCM). Wave mode decomposition shows dominant diurnal tide and important semi‐diurnal tide and diurnal Kelvin wave, with maximal amplitudes of 5, 3, and 2.5 K, respectively, from tens to hundreds of Pa. The results generally agree well with the LMD Mars GCM, but with noticeable earlier phases of diurnal (∼1 hr) and semi‐diurnal (∼3 hr) tides.
The evolution of coherent vortical structures in increasingly turbulent stratified shear layers

Abstract We study the morphology of Eulerian vortical structures and their interaction with density interfaces in increasingly turbulent stably stratified shear layers. We analyse the three-dimensional, simultaneous velocity and density fields obtained in the stratified inclined duct laboratory experiment (SID). We track, across 15 datasets, the evolution of coherent structures from pre-turbulent Holmboe waves, through intermittent turbulence, to full turbulence and mixing. We use the rortex–shear decomposition of the local vorticity vectors into a rortex vector capturing rigid-body rotation and a shear vector. We describe the morphology of ubiquitous hairpin-like vortical structures (revealed by the rortex), similar to those commonly observed in boundary-layer turbulence. These are born as relatively weak vortices around the strong three-dimensional shearing structures of confined Holmboe waves, and gradually strengthen and deform under increasing turbulence, transforming into pairs of upward- and downward-pointing hairpins propagating in opposite directions on the top and bottom edge of the shear layer. The pair of legs for each hairpin are counter-rotating and entrain fluid laterally and vertically, whereas their arched-up ‘heads’, which are transverse vortices, entrain fluid vertically. We then elucidate how this large-scale vortex morphology stirs and mixes the density field. Essentially, vortices located at the sharp density interface on either edge of the mixing layer (mostly hairpin heads) engulf blobs of unmixed fluid into the mixing layer, whereas vortices inside the mixing layer (mostly hairpin legs) further stir it, generating strong, small-scale shear, enhancing mixing. These findings provide new insights into the role of turbulent coherent structures in shear-driven stratified mixing.
Shape Optimization for the Mitigation of Coastal Erosion via Smoothed Particle Hydrodynamics

Adjoint-based shape optimization most often relies on Eulerian flow field formulations. However, since Lagrangian particle methods are the natural choice for solving sedimentation problems in oceanography, extensions to the Lagrangian framework are desirable. For the mitigation of coastal erosion, we perform shape optimization for fluid flows, that are described by Lagrangian shallow water equations and discretized via smoothed particle hydrodynamics. The obstacle's shape is hereby optimized over an appropriate cost function to minimize the height of water waves along the shoreline based on shape calculus. Theoretical results will be numerically verified by exploring different scenarios.
Who Should Review Your Proposal? Interdisciplinary Topic Path Detection for Research Proposals

The peer merit review of research proposals has been the major mechanism to decide grant awards. Nowadays, research proposals have become increasingly interdisciplinary. It has been a longstanding challenge to assign proposals to appropriate reviewers. One of the critical steps in reviewer assignment is to generate accurate interdisciplinary topic labels for proposals. Existing systems mainly collect topic labels manually reported by discipline investigators. However, such human-reported labels can be non-accurate and incomplete. What role can AI play in developing a fair and precise proposal review system? In this evidential study, we collaborate with the National Science Foundation of China to address the task of automated interdisciplinary topic path detection. For this purpose, we develop a deep Hierarchical Interdisciplinary Research Proposal Classification Network (HIRPCN). We first propose a hierarchical transformer to extract the textual semantic information of proposals. We then design an interdisciplinary graph and leverage GNNs to learn representations of each discipline in order to extract interdisciplinary knowledge. After extracting the semantic and interdisciplinary knowledge, we design a level-wise prediction component to fuse the two types of knowledge representations and detect interdisciplinary topic paths for each proposal. We conduct extensive experiments and expert evaluations on three real-world datasets to demonstrate the effectiveness of our proposed model.
Bypassing Markov Chains for Bayesian Generalized Linear Mixed Effects Models

Markov chain Monte Carlo (MCMC) is an all-purpose tool that allows one to generate dependent replicates from a posterior distribution for effectively any Bayesian hierarchical model. As such, MCMC has become a standard in Bayesian statistics. However, convergence issues, tuning, and the effective sample size of the MCMC are nontrivial considerations that are often overlooked or can be difficult to assess. Moreover, these practical issues can produce a significant computational burden. This motivates us to consider finding closed-form expressions of the posterior distribution that are computationally straightforward to sample from directly. We focus on a broad class of Bayesian generalized linear mixed-effects models (GLMM) that allows one to jointly model data of different types (e.g., Gaussian, Poisson, and binomial distributed observations). Exact sampling from the posterior distribution for Bayesian GLMMs is such a difficult problem that it is now arguably overlooked as a possible problem to solve. To solve this problem, we derive a new class of distributions that gives one the flexibility to specify the prior on fixed and random effects to be any conjugate multivariate distribution. We refer to this new distribution as the generalized conjugate multivariate (GCM) distribution, and several technical results are provided. The expression of the exact posterior distribution is given along with the steps to obtain direct independent simulations from the posterior distribution. These direct simulations have an efficient projection/regression form, and hence, we refer to our method as Exact Posterior Regression (EPR). Several theoretical results are developed that create the foundation for EPR. Illustrative examples are provided including a simulation study and an analysis of estimates from the U.S. Census Bureau’s American Community Survey (ACS).
Investigating the Effects of Robot Proficiency Self-Assessment on Trust and Performance

Human-robot teams will soon be expected to accomplish complex tasks in high-risk and uncertain environments. Here, the human may not necessarily be a robotics expert, but will need to establish a baseline understanding of the robot's abilities in order to appropriately utilize and rely on the robot. This willingness to rely, also known as trust, is based partly on the human's belief in the robot's proficiency at a given task. If trust is too high, the human may push the robot beyond its capabilities. If trust is too low, the human may not utilize it when they otherwise could have, wasting precious resources. In this work, we develop and execute an online human-subjects study to investigate how robot proficiency self-assessment reports based on Factorized Machine Self-Confidence affect operator trust and task performance in a grid world navigation task. Additionally we present and analyze a metric for trust level assessment, which measures the allocation of control between an operator and robot when the human teammate is free to switch between teleportation and autonomous control. Our results show that an a priori robot self-assessment report aligns operator trust with robot proficiency, and leads to performance improvements and small increases in self-reported trust.
Learning the travelling salesperson problem requires rethinking generalization

End-to-end training of neural network solvers for graph combinatorial optimization problems such as the Travelling Salesperson Problem (TSP) have seen a surge of interest recently, but remain intractable and inefficient beyond graphs with few hundreds of nodes. While state-of-the-art learning-driven approaches for TSP perform closely to classical solvers when trained on trivially small sizes, they are unable to generalize the learnt policy to larger instances at practical scales. This work presents an end-to-end neural combinatorial optimization pipeline that unifies several recent papers in order to identify the inductive biases, model architectures and learning algorithms that promote generalization to instances larger than those seen in training. Our controlled experiments provide the first principled investigation into such zero-shot generalization, revealing that extrapolating beyond training data requires rethinking the neural combinatorial optimization pipeline, from network layers and learning paradigms to evaluation protocols. Additionally, we analyze recent advances in deep learning for routing problems through the lens of our pipeline and provide new directions to stimulate future research.
Efficient Remote Photoplethysmography with Temporal Derivative Modules and Time-Shift Invariant Loss

We present a lightweight neural model for remote heart rate estimation focused on the efficient spatio-temporal learning of facial photoplethysmography (PPG) based on i) modelling of PPG dynamics by combinations of multiple convolutional derivatives, and ii) increased flexibility of the model to learn possible offsets between the facial video PPG and the ground truth. PPG dynamics are modelled by a Temporal Derivative Module (TDM) constructed by the incremental aggregation of multiple convolutional derivatives, emulating a Taylor series expansion up to the desired order. Robustness to ground truth offsets is handled by the introduction of TALOS (Temporal Adaptive LOcation Shift), a new temporal loss to train learning-based models. We verify the effectiveness of our model by reporting accuracy and efficiency metrics on the public PURE and UBFC-rPPG datasets. Compared to existing models, our approach shows competitive heart rate estimation accuracy with a much lower number of parameters and lower computational cost.