Miracle-dz/UniFAR-FTUs huggingface dataset:
Gaseous dynamical friction under radiative feedback: do intermediate-mass black holes speed up or down?

Coalescence of intermediate-mass black holes (IMBHs) as a result of the migration toward galactic centers via dynamical friction may contribute to the formation of supermassive BHs. Here we reinvestigate the gaseous dynamical friction, which was claimed to be inefficient with radiative feedback from BHs in literature, by performing 3D radiation-hydrodynamics simulations that solve the flow structure in the vicinity of BHs. We consider a $10^4~M_\odot$ BH moving at the velocity $V_{\rm flow}$ through the homogeneous medium with metallicity $Z$ in the range of $0-0.1~Z_\odot$ and density $n_{\infty}$. We show that, if $n_{\infty} \lesssim 10^{6}~{\rm cm^{-3}}$ and $V_{\rm flow} \lesssim 60~{\rm km~s^{-1}}$, the BH is accelerated forward because of the gravitational pull from a dense shell ahead of an ionized bubble around the BH, regardless of the value of $Z$. If $n_{\infty} \gtrsim 10^{6}~{\rm cm^{-3}}$, however, our simulation shows the opposite result. The ionized bubble and associating shell temporarily appear, but immediately go downstream with significant ram pressure of the flow. They eventually converge into a massive downstream wake, which gravitationally drags the BH backward. The BH decelerates over the timescale of $\sim 0.01$~Myr, much shorter than the dynamical timescale in galactic disks. Our results suggest that IMBHs that encounter the dense clouds rapidly migrate toward galactic centers, where they possibly coalescence with others.
Closed-Form Optimal Impulsive Control of Spacecraft Relative Motion Using Reachable Set Theory

This paper addresses the spacecraft relative orbit reconfiguration problem of minimizing the delta-v cost of impulsive control actions while achieving a desired state in fixed time. The problem is posed in relative orbit element (ROE) space, which yields insight into relative motion geometry and allows for the straightforward inclusion of perturbations in linear time-variant form. Reachable set theory is used to translate the cost-minimization problem into a geometric path-planning problem and formulate the reachable delta-v minimum, a new metric to assess optimality and quantify reachability of a maneuver scheme. Next, this paper presents a methodology to compute maneuver schemes that meet this new optimality criteria and achieve a prescribed reconfiguration. Though the methodology is applicable to any linear time-variant system, this paper leverages a state representation in ROE to derive new globally optimal maneuver schemes in orbits of arbitrary eccentricity. The methodology is also used to generate quantifiably sub-optimal solutions when the optimal solutions are unreachable. Further, this paper determines the mathematical impact of uncertainties on achieving the desired end state and provides a geometric visualization of those effects on the reachable set. The proposed algorithms are tested in realistic reconfiguration scenarios and validated in a high-fidelity simulation environment.
Learning Global Transparent Models from Local Contrastive Explanations

There is a rich and growing literature on producing local contrastive/counterfactual explanations for black-box models (e.g. neural networks). In these methods, for an input, an explanation is in the form of a contrast point differing in very few features from the original input and lying in a different class. Other works try to build globally interpretable models like decision trees and rule lists based on the data using actual labels or based on the black-box models predictions. Although these interpretable global models can be useful, they may not be consistent with local explanations from a specific black-box of choice. In this work, we explore the question: Can we produce a transparent global model that is simultaneously accurate and consistent with the local (contrastive) explanations of the black-box model? We introduce a natural local consistency metric that quantifies if the local explanations and predictions of the black-box model are also consistent with the proxy global transparent model. Based on a key insight we propose a novel method where we create custom boolean features from \emph{only} (sparse) local contrastive explanations of the black-box model and then train a globally transparent model on just these, and showcase empirically that such models have higher local consistency compared with other known strategies, while still being close in performance to models that are trained with access to the original data.
Incremental Nonlinear Fault-Tolerant Control of a Quadrotor With Complete Loss of Two Opposing Rotors

In order to further expand the flight envelope of quadrotors under actuator failures, we design a nonlinear sensor-based fault-tolerant controller to stabilize a quadrotor with failure of two opposing rotors in the high-speed flight condition (>8 m/s). The incremental nonlinear dynamic inversion approach which excels in handling model uncertainties is adopted to compensate for the significant unknown aerodynamic effects. The internal dynamics of such an underactuated system have been analyzed, and subsequently stabilized by redefining the control output. The proposed method can be generalized to control a quadrotor under single-rotor-failure and nominal conditions. For validation, flight tests have been carried out in a large-scale open jet wind tunnel. The position of a damaged quadrotor can be controlled in the presence of significant wind disturbances. A linear quadratic regulator approach from the literature has been compared to demonstrate the advantages of the proposed nonlinear method in the windy and high-speed flight condition.
Empirical Policy Evaluation With Supergraphs

We devise algorithms for the policy evaluation problem in reinforcement learning, assuming access to a simulator and certain side information called the <italic>supergraph</italic>. Our algorithms explore backward from high-cost states to find high-value ones, in contrast to approaches that work forward from all states. While several papers have demonstrated the utility of backward exploration empirically, we conduct rigorous analyses which show that our algorithms can reduce average-case sample complexity from <inline-formula> <tex-math notation="LaTeX">$O(S \log S)$ </tex-math></inline-formula> to as low as <inline-formula> <tex-math notation="LaTeX">$O(\log S)$ </tex-math></inline-formula>. Analytically, we adapt tools from the network science literature to provide a new methodology for reinforcement learning problems.
Tree-structured Attention with Hierarchical Accumulation

Incorporating hierarchical structures like constituency trees has been shown to be effective for various natural language processing (NLP) tasks. However, it is evident that state-of-the-art (SOTA) sequence-based models like the Transformer struggle to encode such structures inherently. On the other hand, dedicated models like the Tree-LSTM, while explicitly modeling hierarchical structures, do not perform as efficiently as the Transformer. In this paper, we attempt to bridge this gap with Hierarchical Accumulation to encode parse tree structures into self-attention at constant time complexity. Our approach outperforms SOTA methods in four IWSLT translation tasks and the WMT'14 English-German task. It also yields improvements over Transformer and Tree-LSTM on three text classification tasks. We further demonstrate that using hierarchical priors can compensate for data shortage, and that our model prefers phrase-level attentions over token-level attentions.
Span Recovery for Deep Neural Networks with Applications to Input Obfuscation

The tremendous success of deep neural networks has motivated the need to better understand the fundamental properties of these networks, but many of the theoretical results proposed have only been for shallow networks. In this paper, we study an important primitive for understanding the meaningful input space of a deep network: span recovery. For $k<n$, let $\mathbf{A} \in \mathbb{R}^{k \times n}$ be the innermost weight matrix of an arbitrary feed forward neural network $M:\mathbb{R}^n \to \mathbb{R}$, so $M(x)$ can be written as $M(x) = \sigma(\mathbf{A} x)$, for some network $\sigma:\mathbb{R}^k \to \mathbb{R}$. The goal is then to recover the row span of $\mathbf{A}$ given only oracle access to the value of $M(x)$. We show that if $M$ is a multi-layered network with ReLU activation functions, then partial recovery is possible: namely, we can provably recover $k/2$ linearly independent vectors in the row span of $\mathbf{A}$ using poly$(n)$ non-adaptive queries to $M(x)$. Furthermore, if $M$ has differentiable activation functions, we demonstrate that full span recovery is possible even when the output is first passed through a sign or $0/1$ thresholding function; in this case our algorithm is adaptive. Empirically, we confirm that full span recovery is not always possible, but only for unrealistically thin layers. For reasonably wide networks, we obtain full span recovery on both random networks and networks trained on MNIST data. Furthermore, we demonstrate the utility of span recovery as an attack by inducing neural networks to misclassify data obfuscated by controlled random noise as sensical inputs.
Fair Clustering with Multiple Colors

A fair clustering instance is given a data set $A$ in which every point is assigned some color. Colors correspond to various protected attributes such as sex, ethnicity, or age. A fair clustering is an instance where membership of points in a cluster is uncorrelated with the coloring of the points. Of particular interest is the case where all colors are equally represented. If we have exactly two colors, Chierrichetti, Kumar, Lattanzi and Vassilvitskii (NIPS 2017) showed that various $k$-clustering objectives admit a constant factor approximation. Since then, a number of follow up work has attempted to extend this result to a multi-color case, though so far, the only known results either result in no-constant factor approximation, apply only to special clustering objectives such as $k$-center, yield bicrititeria approximations, or require $k$ to be constant. In this paper, we present a simple reduction from unconstrained $k$-clustering to fair $k$-clustering for a large range of clustering objectives including $k$-median, $k$-means, and $k$-center. The reduction loses only a constant factor in the approximation guarantee, marking the first true constant factor approximation for many of these problems.
SPORES

Machine learning algorithms are commonly specified in linear algebra (LA). LA expressions can be rewritten into more efficient forms, by taking advantage of input properties such as sparsity, as well as program properties such as common subexpressions and fusible operators. The complex interaction among these properties' impact on the execution cost poses a challenge to optimizing compilers. Existing compilers resort to intricate heuristics that complicate the codebase and add maintenance cost, but fail to search through the large space of equivalent LA expressions to find the cheapest one. We introduce a general optimization technique for LA expressions, by converting the LA expressions into Relational Algebra (RA) expressions, optimizing the latter, then converting the result back to (optimized) LA expressions. The rewrite rules we design in this approach are complete, meaning that any equivalent LA expression is covered in the search space. The challenge is the major size of the search space, and we address this by adopting and extending a technique used in compilers, called equality saturation. Our optimizer, SPORES, uses rule sampling to quickly cover vast portions of the search space; it then uses a constraint solver to extract the optimal plan from the covered space, or alternatively uses a greedy algorithm to shorten compile time. We integrate SPORES into SystemML and validate it empirically across a spectrum of machine learning tasks; SPORES can derive all existing hand-coded optimizations in SystemML, and perform new optimizations that lead to up to 10X speedup.
Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology

Recent works have shown that gradient descent can find a global minimum for over-parameterized neural networks where the widths of all the hidden layers scale polynomially with $N$ ($N$ being the number of training samples). In this paper, we prove that, for deep networks, a single layer of width $N$ following the input layer suffices to ensure a similar guarantee. In particular, all the remaining layers are allowed to have constant widths, and form a pyramidal topology. We show an application of our result to the widely used Xavier's initialization and obtain an over-parameterization requirement for the single wide layer of order $N^2. $
A strong law of large numbers for scrambled net integration

This article provides a strong law of large numbers for integration on digital nets randomized by a nested uniform scramble. The motivating problem is optimization over some variables of an integral over others, arising in Bayesian optimization. This strong law requires that the integrand have a finite moment of order $p$ for some $p>1$. Previously known results implied a strong law only for Riemann integrable functions. Previous general weak laws of large numbers for scrambled nets require a square integrable integrand. We generalize from $L^2$ to $L^p$ for $p>1$ via the Riesz-Thorin interpolation theorem
Linear Regression Models in Epidemiology

The paper proposes to analyze epidemiological data using regression models which enable subject-matter (epidemiological) interpretation of such data whether with uncorrelated or correlated predictors. To this end, response functions should include not only terms linear in predictors but also higher order ones (e.g. quadratic and cross terms). For epidemiological interpretation of a regression model, the suggestion is to construct conditional functions derived from the general regression function with the values of all predictor variables held fixed excepting one predictor. Unlike the conventional techniques based on linear-predictor models in which the coefficient at any variable is interpreted, our approach proposes to interpret this conditional function, which is multivariate for any predictor being dependent on the values of all the other predictors. It is such functions that can describe relationships between Y and a predictor that have different forms in different predictor domains. The paper discusses differences in the interpretation of the proposed conditional functions between cases involving correlated and uncorrelated predictor variables. The construction and analysis of regression models for epidemiological and environmental data are illustrated with examples.
Learning When to Transfer among Agents: An Efficient Multiagent Transfer Learning Framework

Transfer Learning has shown great potential to enhance the single-agent Reinforcement Learning (RL) efficiency, by sharing previously learned policies. Inspired by this, the team learning performance in multiagent settings can be potentially promoted with agents reusing knowledge between each other when all agents interact with the environment and learn simultaneously. However, how each independent agent selectively learns from other agents' knowledge is still a problem. In this paper, we propose a novel multi-agent transfer learning framework to improve the learning efficiency of multiagent systems. Our framework learns when and what advice to give to each agent and when to terminate it by modeling multi-agent transfer as the option learning problem. We also propose a novel option learning algorithm, named as the Successor Representation Option (SRO) learning that decouples the dynamics of the environment from the rewards to learn the option-value function under each agent's preference. The proposed framework can be easily combined with existing deep RL approaches. Experimental results show it significantly accelerates the learning process and surpasses state-of-the-art deep RL methods in terms of learning efficiency and final performance in both discrete and continuous action spaces.
Joint Data and Active User Detection for Grant-free FTN-NOMA in Dynamic Networks

Both faster than Nyquist (FTN) signaling and non-orthogonal multiple access (NOMA) are promising next generation wireless communications techniques as a benefit of their capability of improving the system’s spectral efficiency. This paper considers an uplink system that combines the advantages of FTN and NOMA. Consequently, an improved spectral efficiency is achieved by deliberately introducing both inter-symbol interference (ISI) and inter-user interference (IUI). More specifically, we propose a grant-free transmission scheme to reduce the signaling overhead and transmission latency of the considered NOMA system. To distinguish the active and inactive users, we develop a novel message passing receiver that jointly estimates the channel state, detects the user activity, and performs decoding. We conclude by quantifying the significant spectral efficiency gain achieved by our amalgamated FTN-NOMA scheme compared to the orthogonal transmission system, which is up to 87.5%.
An improved nonlinear FastEuler AHRS estimation based on the SVDCKF algorithm

In this paper, we present a Singular Value Decomposition Cubature Kalman Filter(SVDCKF) fusion algorithm based on the improved nonlinear FastEuler Attitude and Heading Reference and System(AHRS) estimation model for small-UAV attitude. The contributions of this work are the derivation of the low-cost IMU/MAG integrated AHRS model combined with the quaternion attitude determination, and use the FastEuler to correct the gyroscope attitude update, which can increase the real-time solution. In addition, the SVDCKF algorithm is fused the various raw sensors data in order to improve the filter accuracy compared with the CKF. The simulation and experiment results demonstrate the proposed algorithm has the more excellent attitude solution accuracy compared with the CKF in the low and high dynamic flight conditions.
LAMBERT: Layout-Aware language Modeling using BERT for information extraction

In this paper we introduce a novel approach to the problem of understanding documents where the local semantics is influenced by non-trivial layout. Namely, we modify the Transformer architecture in a way that allows it to use the graphical features defined by the layout, without the need to re-learn the language semantics from scratch, thanks to starting the training process from a model pretrained on classical language modeling tasks.
Emotion Recognition Through Observer's Physiological Signals

Emotion recognition based on physiological signals is a hot topic and has a wide range of applications, like safe driving, health care and creating a secure society. This paper introduces a physiological dataset PAFEW, which is obtained using movie clips from the Acted Facial Expressions in the Wild (AFEW) dataset as stimuli. To establish a baseline, we use the electrodermal activity (EDA) signals in this dataset and extract 6 features from each signal series corresponding to each movie clip to recognize 7 emotions, i.e., Anger, Disgust, Fear, Happy, Surprise, Sad and Neutral. Overall, 24 observers participated in our collection of the training set, including 19 observers who participated in only one session watching 80 videos from 7 classes and 5 observers who participated multiple times and watched all the videos. All videos were presented in an order balanced fashion. Leave-one-observer-out was employed in this classification task. We report the classification accuracy of our baseline, a three-layer network, on this initial training set while training with signals from all participants, only single participants and only multiple participants. We also investigate the recognition accuracy of grouping the dataset by arousal or valence, which achieves 68.66% and 72.72% separately. Finally, we provide a two-step network. The first step is to classify the features into high/low arousal or positive/negative valence by a network. Then the arousal/valence middle output of the first step is concatenated with feature sets as input of the second step for emotion recognition. We found that adding arousal or valence information can help to improve the classification accuracy. In addition, the information of positive/negative valence boosts the classification accuracy to a higher degree on this dataset.
Interpreting Interpretations: Organizing Attribution Methods by Criteria

Motivated by distinct, though related, criteria, a growing number of attribution methods have been developed to interprete deep learning. While each relies on the interpretability of the concept of "importance" and our ability to visualize patterns, explanations produced by the methods often differ. In this work we expand the foundations of human-understandable concepts with which attributions can be interpreted beyond "importance" and its visualization; we incorporate the logical concepts of necessity and sufficiency, and the concept of proportionality. We define metrics to represent these concepts as quantitative aspects of an attribution. We evaluate our measures on a collection of methods explaining convolutional neural networks (CNN) for image classification. We conclude that some attribution methods are more appropriate for interpretation in terms of necessity while others are in terms of sufficiency, while no method is always the most appropriate in terms of both.
CNN-based approach for glaucoma diagnosis using transfer learning and LBP-based data augmentation

Glaucoma causes an irreversible damage to retinal nerve fibers which results in vision loss, if undetected in early stage. Therefore, diagnosis of glaucoma in its early stage may prevent further vision loss. In this paper, we propose a convolutional neural network (CNN) based approach for automated glaucoma diagnosis by employing retinal fundus images. This approach employs transfer learning technique and local binary pattern (LBP) based data augmentation. In the proposed approach, we employ Alexnet as a pre-trained CNN model which is used for transfer learning. Initially, the proposed approach divides the fundus image dataset into training and testing data. Further, the color fundus images in training and testing data are separated into red (R), green (G), and blue (B) channels. Additionally, the LBP-based data augmentation is performed on training data. Specifically, we compute LPBs for each of the channel. Finally, the augmented training data is used to train the CNN model via transfer learning. In testing stage, the R, G, and B channels of test image are fed to the trained CNN model which generates 3 decisions. We employ a decision level fusion technique to combine the decisions obtained from the trained CNN model. The experimental evaluation of the proposed approach on the public RIM-ONE fundus image database, achieves state-of-the-art performance for glaucoma diagnosis.