Miracle-dz/UniFAR-FTUs huggingface dataset:
Grafting Vision Transformers

Vision Transformers (ViTs) have recently become the state-of-the-art across many computer vision tasks. In contrast to convolutional networks (CNNs), ViTs enable global information sharing even within shallow layers of a network, i.e., among high-resolution features. However, this perk was later overlooked with the success of pyramid architectures such as Swin Transformer, which show better performance-complexity trade-offs. In this paper, we present a simple and efficient add-on component (termed GrafT) that considers global dependencies and multi-scale information throughout the network, in both high- and low-resolution features alike. It has the flexibility of branching out at arbitrary depths and shares most of the parameters and computations of the backbone. GrafT shows consistent gains over various well-known models which includes both hybrid and pure Transformer types, both homogeneous and pyramid structures, and various self-attention methods. In particular, it largely benefits mobile-size models by providing high-level semantics. On the ImageNet-1k dataset, GrafT delivers +3.9%, +1.4%, and +1.9% top-1 accuracy improvement to DeiT-T, Swin-T, and MobileViT-XXS, respectively. Our code and models will be made available.
Multitrack Music Transformer

Existing approaches for generating multitrack music with transformer models have been limited in terms of the number of instruments, the length of the music segments and slow inference. This is partly due to the memory requirements of the lengthy input sequences necessitated by existing representations. In this work, we propose a new multitrack music representation that allows a diverse set of instruments while keeping a short sequence length. Our proposed Multitrack Music Transformer (MMT) achieves comparable performance with state-of-the-art systems, landing in between two recently proposed models in a subjective listening test, while achieving substantial speedups and memory reductions over both, making the method attractive for real time improvisation or near real time creative applications. Further, we propose a new measure for analyzing musical self-attention and show that the trained model attends more to notes that form a consonant interval with the current note and to notes that are 4N beats away from the current step.
Cross sections from 5 to 35 MeV for the reactions natMg(3He,x)26Al, 27Al(3He,x)26Al, natCa(3He,x)41Ca, and natCa(3He,x)36Cl: Implications for early irradiation in the solar system

Abstract– Cross sections were measured for the nuclear reactions natMg(3He,x)26Al, 27Al(3He,x)26Al, natCa(3He,x)41Ca, and natCa(3He,x)36Cl in the energy region from approximately 5–35 MeV. The rates of these reactions are important for studies of early solar system irradiation scenarios. The 26Al, 36Cl, and 41Ca were separated chemically, and the numbers of atoms produced in each reaction channel were measured using accelerator mass spectrometry (AMS). From these results, 26 cross sections were determined and compared with predictions of the TALYS code. Agreement is within 40% for most cross sections. Our measurements were used to model the production of 7Be, 10Be, 26Al, and 41Ca in the early solar system. For projectiles 1H, 3He, and 4He, we assumed energy spectra of the general form E−α. For a wide range of parameterizations, the modeled ratios of 7Be/Be and 10Be/Be on the one hand and of 26Al/27Al and 41Ca/Ca on the other are coupled because the excitation functions for the relevant nuclear reactions have similar shapes. Modeling of a closed system with the constraint that 10Be/9Be = 0.001 fails to reproduce simultaneously the range of 7Be/9Be, 26Al/27Al, and 41Ca/Ca ratios inferred for the early solar system from studies of meteorites.
Driving gas shells with radiation pressure on dust in radiation-hydrodynamic simulations

We present radiation-hydrodynamic simulations of radiatively-driven gas shells launched by bright active galactic nuclei (AGN) in isolated dark matter haloes. Our goals are (1) to investigate the ability of AGN radiation pressure on dust to launch galactic outflows and (2) to constrain the efficiency of infrared (IR) multi-scattering in boosting outflow acceleration. Our simulations are performed with the radiation-hydrodynamic code RAMSES-RT and include both single- and multi-scattered radiation pressure from an AGN, radiative cooling and self-gravity. Since outflowing shells always eventually become transparent to the incident radiation field, outflows that sweep up all intervening gas are likely to remain gravitationally bound to their halo even at high AGN luminosities. The expansion of outflowing shells is well described by simple analytic models as long as the shells are mildly optically thick to IR radiation. In this case, an enhancement in the acceleration of shells through IR multi-scattering occurs as predicted, i.e. a force dP/dt = tau_IR L/c is exerted on the gas. For high optical depths tau_IR > 50, however, momentum transfer between outflowing optically thick gas and IR radiation is rapidly suppressed, even if the radiation is efficiently confined. At high tau_IR, the characteristic flow time becomes shorter than the required trapping time of IR radiation such that the momentum flux dP/dt << tau_IR L/c. We argue that while unlikely to unbind massive galactic gaseous haloes, AGN radiation pressure on dust could play an important role in regulating star formation and black hole accretion in the nuclei of massive compact galaxies at high redshift.
Nonuniqueness and Convergence to Equivalent Solutions in Observer-based Inverse Reinforcement Learning

A key challenge in solving the deterministic inverse reinforcement learning problem online and in real-time is the existence of non-unique solutions. Nonuniqueness necessitates the study of the notion of equivalent solutions and convergence to such solutions. While offline algorithms that result in convergence to equivalent solutions have been developed in the literature, online, real-time techniques that address nonuniqueness are not available. In this paper, a regularized history stack observer is developed to generate solutions that are approximately equivalent. Novel data-richness conditions are developed to facilitate the analysis and simulation results are provided to demonstrate the effectiveness of the developed technique.
Self-Supervised Learning with Multi-View Rendering for 3D Point Cloud Analysis

Recently, great progress has been made in 3D deep learning with the emergence of deep neural networks specifically designed for 3D point clouds. These networks are often trained from scratch or from pre-trained models learned purely from point cloud data. Inspired by the success of deep learning in the image domain, we devise a novel pre-training technique for better model initialization by utilizing the multi-view rendering of the 3D data. Our pre-training is self-supervised by a local pixel/point level correspondence loss computed from perspective projection and a global image/point cloud level loss based on knowledge distillation, thus effectively improving upon popular point cloud networks, including PointNet, DGCNN and SR-UNet. These improved models outperform existing state-of-the-art methods on various datasets and downstream tasks. We also analyze the benefits of synthetic and real data for pre-training, and observe that pre-training on synthetic data is also useful for high-level downstream tasks. Code and pre-trained models are available at https://github.com/VinAIResearch/selfsup_pcd.
Scalar fields and Lifshitz black holes from Derrick’s theorem evasion

We study classical scalar ﬁelds in asymptotically Lifshitz spacetimes. By evading Derrick’s theorem requiring the scalar potential to explicitly depend on the background coordinates, we induce a diﬀeomorphism invariance breaking and settle a formalism to ﬁnd spatially localized solutions in the probe ﬁeld limit. By inserting a backreaction equipped with a cosmological constant and a Maxwell term coupled to a dielectric function in the model, we ﬁnd a Lifshitz black hole solution
Modeling structure-building in the brain with CCG parsing and large language models

To model behavioral and neural correlates of language comprehension in naturalistic environments, researchers have turned to broad-coverage tools from natural-language processing and machine learning. Where syntactic structure is explicitly modeled, prior work has relied predominantly on context-free grammars (CFGs), yet such formalisms are not sufficiently expressive for human languages. Combinatory categorial grammars (CCGs) are sufficiently expressive directly compositional models of grammar with flexible constituency that affords incremental interpretation. In this work, we evaluate whether a more expressive CCG provides a better model than a CFG for human neural signals collected with functional magnetic resonance imaging (fMRI) while participants listen to an audiobook story. We further test between variants of CCG that differ in how they handle optional adjuncts. These evaluations are carried out against a baseline that includes estimates of next-word predictability from a transformer neural network language model. Such a comparison reveals unique contributions of CCG structure-building predominantly in the left posterior temporal lobe: CCG-derived measures offer a superior fit to neural signals compared to those derived from a CFG. These effects are spatially distinct from bilateral superior temporal effects that are unique to predictability. Neural effects for structure-building are thus separable from predictability during naturalistic listening, and those effects are best characterized by a grammar whose expressive power is motivated on independent linguistic grounds.
Dysfluencies Seldom Come Alone - Detection as a Multi-Label Problem

Specially adapted speech recognition models are necessary to handle stuttered speech. For these to be used in a targeted manner, stuttered speech must be reliably detected. Recent works have treated stuttering as a multi-class classiﬁcation problem or viewed detecting each dysﬂuency type as an iso-lated task; that does not capture the nature of stuttering, where one dysﬂuency seldom comes alone, i.e., co-occurs with oth-ers. This work explores an approach based on a modiﬁed wav2vec 2.0 system for end-to-end stuttering detection and classiﬁcation as a multi-label problem. The method is evaluated on combinations of three datasets containing English and German stuttered speech, yielding state-of-the-art results for stuttering detection on the SEP-28k-Extended dataset. Ex-perimental results provide evidence for the transferability of features and the generalizability of the method across datasets and languages.
Multivariate Permutation Tests : With Applications in Biostatistics

Preface. Notation and Abbreviations. Introduction. Discussion of a Simple Testing Problem. Theory of Permutation Tests for One-Sample Problems. Examples of Univariate Multi-Sample Problems. Theory of Permutation Tests for Multi-Sample Problems. Nonparametric Combination Methodology. Examples of Nonparametric Combination. Permutation Analysis in Factorial Designs. Permutation Testing with Missing Data. The Behrens-Fisher Permutation Problem. Permutation Testing for Repeated Measurements. Further Applications. References. Index
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers

Large pre-trained language models have shown remarkable performance over the past few years. These models, however, sometimes learn superficial features from the dataset and cannot generalize to the distributions that are dissimilar to the training scenario. There have been several approaches proposed to reduce model's reliance on these bias features which can improve model robustness in the out-of-distribution setting. However, existing methods usually use a fixed low-capacity model to deal with various bias features, which ignore the learnability of those features. In this paper, we analyze a set of existing bias features and demonstrate there is no single model that works best for all the cases. We further show that by choosing an appropriate bias model, we can obtain a better robustness result than baselines with a more sophisticated model design.
Coverage-centric Coreset Selection for High Pruning Rates

One-shot coreset selection aims to select a representative subset of the training data, given a pruning rate, that can later be used to train future models while retaining high accuracy. State-of-the-art coreset selection methods pick the highest importance examples based on an importance metric and are found to perform well at low pruning rates. However, at high pruning rates, they suffer from a catastrophic accuracy drop, performing worse than even random sampling. This paper explores the reasons behind this accuracy drop both theoretically and empirically. We first propose a novel metric to measure the coverage of a dataset on a specific distribution by extending the classical geometric set cover problem to a distribution cover problem. This metric helps explain why coresets selected by SOTA methods at high pruning rates perform poorly compared to random sampling because of worse data coverage. We then propose a novel one-shot coreset selection method, Coverage-centric Coreset Selection (CCS), that jointly considers overall data coverage upon a distribution as well as the importance of each example. We evaluate CCS on five datasets and show that, at high pruning rates (e.g., 90%), it achieves significantly better accuracy than previous SOTA methods (e.g., at least 19.56% higher on CIFAR10) as well as random selection (e.g., 7.04% higher on CIFAR10) and comparable accuracy at low pruning rates. We make our code publicly available at https://github.com/haizhongzheng/Coverage-centric-coreset-selection.
Interactive Imitation Learning of Bimanual Movement Primitives

Performing bimanual tasks with dual robotic setups can drastically increase the impact on industrial and daily life applications. However, performing a bimanual task brings many challenges, like synchronization and coordination of the single-arm policies. This article proposes the Safe, Interactive Movement Primitives Learning (SIMPLe) algorithm, to teach and correct single or dual arm impedance policies directly from human kinesthetic demonstrations. Moreover, it proposes a novel graph encoding of the policy based on Gaussian Process Regression (GPR) where the single-arm motion is guaranteed to converge close to the trajectory and then towards the demonstrated goal. Regulation of the robot stiffness according to the epistemic uncertainty of the policy allows for easily reshaping the motion with human feedback and/or adapting to external perturbations. We tested the SIMPLe algorithm on a real dual-arm setup where the teacher gave separate single-arm demonstrations and then successfully synchronized them only using kinesthetic feedback or where the original bimanual demonstration was locally reshaped to pick a box at a different height.
Incorporating Interactive Facts for Stock Selection via Neural Recursive ODEs

—Stock selection attempts to rank a list of stocks for optimizing investment decision making, aiming at minimizing investment risks while maximizing proﬁt returns. Recently, researchers have developed various (recurrent) neural network-based methods to tackle this problem. Without exceptions, they primarily leverage historical market volatility to enhance the selection performance. However, these approaches greatly rely on discrete sampled market observations, which either fail to consider the uncertainty of stock ﬂuctuations or predict continuous stock dynamics in the future. Besides, some studies have considered the explicit stock interdependence derived from multiple domains (e.g., industry and shareholder). Nevertheless, the implicit cross-dependencies among different domains are under-explored. To address such limitations, we present a novel stock selection solution – StockODE, a latent variable model with Gaussian prior. Speciﬁcally, we devise a Movement Trend Correlation module to expose the time-varying relationships regarding stock movements. We design Neural Recursive Ordinary Differential Equation Networks (NRODEs) to capture the temporal evolution of stock volatility in a continuous dynamic manner. Moreover, we build a hierarchical hypergraph to incorporate the domain-aware dependencies among the stocks. Experiments conducted on two real-world stock market datasets demonstrate that StockODE signiﬁcantly outperforms several baselines, such as up to 18.57% average improvement regarding Sharpe Ratio.
Layer-wise Shared Attention Network on Dynamical System Perspective

Attention networks have successfully boosted accuracy in various vision problems. Previous works lay emphasis on designing a new self-attention module and follow the traditional paradigm that individually plugs the modules into each layer of a network. However, such a paradigm inevitably increases the extra parameter cost with the growth of the number of layers. From the dynamical system perspective of the residual neural network, we find that the feature maps from the layers of the same stage are homogenous, which inspires us to propose a novel-and-simple framework, called the dense and implicit attention (DIA) unit, that shares a single attention module throughout different network layers. With our framework, the parameter cost is independent of the number of layers and we further improve the accuracy of existing popular self-attention modules with significant parameter reduction without any elaborated model crafting. Extensive experiments on benchmark datasets show that the DIA is capable of emphasizing layer-wise feature interrelation and thus leads to significant improvement in various vision tasks, including image classification, object detection, and medical application. Furthermore, the effectiveness of the DIA unit is demonstrated by novel experiments where we destabilize the model training by (1) removing the skip connection of the residual neural network, (2) removing the batch normalization of the model, and (3) removing all data augmentation during training. In these cases, we verify that DIA has a strong regularization ability to stabilize the training, i.e., the dense and implicit connections formed by our method can effectively recover and enhance the information communication across layers and the value of the gradient thus alleviate the training instability.
Nearest Neighbor Language Models for Stylistic Controllable Generation

Recent language modeling performance has been greatly improved by the use of external memory. This memory encodes the context so that similar contexts can be recalled during decoding. This similarity depends on how the model learns to encode context, which can be altered to include other attributes, such as style. We construct and evaluate an architecture for this purpose, using corpora annotated for politeness, formality, and toxicity. Through extensive experiments and human evaluation we demonstrate the potential of our method to generate text while controlling style. We find that style-specific datastores improve generation performance, though results vary greatly across styles, and the effect of pretraining data and specific styles should be explored in future work.
Adapting Neural Models with Sequential Monte Carlo Dropout

The ability to adapt to changing environments and settings is essential for robots acting in dynamic and unstructured environments or working alongside humans with varied abilities or preferences. This work introduces an extremely simple and effective approach to adapting neural models in response to changing settings. We first train a standard network using dropout, which is analogous to learning an ensemble of predictive models or distribution over predictions. At run-time, we use a particle filter to maintain a distribution over dropout masks to adapt the neural model to changing settings in an online manner. Experimental results show improved performance in control problems requiring both online and look-ahead prediction, and showcase the interpretability of the inferred masks in a human behaviour modelling task for drone teleoperation.
MagicMix: Semantic Mixing with Diffusion Models

Have you ever imagined what a corgi-alike coffee machine or a tiger-alike rabbit would look like? In this work, we attempt to answer these questions by exploring a new task called semantic mixing, aiming at blending two different semantics to create a new concept (e.g., corgi + coffee machine -->corgi-alike coffee machine). Unlike style transfer, where an image is stylized according to the reference style without changing the image content, semantic blending mixes two different concepts in a semantic manner to synthesize a novel concept while preserving the spatial layout and geometry. To this end, we present MagicMix, a simple yet effective solution based on pre-trained text-conditioned diffusion models. Motivated by the progressive generation property of diffusion models where layout/shape emerges at early denoising steps while semantically meaningful details appear at later steps during the denoising process, our method first obtains a coarse layout (either by corrupting an image or denoising from a pure Gaussian noise given a text prompt), followed by injection of conditional prompt for semantic mixing. Our method does not require any spatial mask or re-training, yet is able to synthesize novel objects with high fidelity. To improve the mixing quality, we further devise two simple strategies to provide better control and flexibility over the synthesized content. With our method, we present our results over diverse downstream applications, including semantic style transfer, novel object synthesis, breed mixing, and concept removal, demonstrating the flexibility of our method. More results can be found on the project page https://magicmix.github. io
Axisymmetric magnetic modes of neutron stars having mixed poloidal and toroidal magnetic fields

We calculate axisymmetric magnetic modes of a neutron star possessing a mixed poloidal and toroidal magnetic field, where the toroidal field is assumed to be proportional to a dimensionless parameter $\zeta_0$. Here, we assume an isentropic structure for the neutron star and consider no effects of rotation. Ignoring the equilibrium deformation due to the magnetic field, we employ a polytrope of the index $n=1$ as the background model for our modal analyses. For the mixed poloidal and toroidal magnetic field with $\zeta_0\not=0$, axisymmetric spheroidal and toroidal modes are coupled. We compute axisymmetric spheroidal and toroidal magnetic modes as a function of the parameter $\zeta_0$ from $0$ to $\sim 1$ for the surface field strengths $B_S=10^{14}$G and $10^{15}$G. We find that the frequency $\omega$ of the magnetic modes decreases with increasing $\zeta_0$. We also find that the frequency of the spheroidal magnetic modes is almost exactly proportional to $B_S$ for $\zeta_0\lesssim 1$ but that this proportionality holds only when $\zeta_0\ll 1$ for the toroidal magnetic modes. The wave patterns of the spheroidal magnetic modes and toroidal magnetic modes are not strongly affected by the coupling so long as $\zeta_0\lesssim 1$. We find no unstable modes having $\omega^2<0$.