Miracle-dz/UniFAR-FTUs huggingface dataset:
Motion Transformer with Global Intention Localization and Local Movement Refinement

Predicting multimodal future behavior of traffic participants is essential for robotic vehicles to make safe decisions. Existing works explore to directly predict future trajectories based on latent features or utilize dense goal candidates to identify agent's destinations, where the former strategy converges slowly since all motion modes are derived from the same feature while the latter strategy has efficiency issue since its performance highly relies on the density of goal candidates. In this paper, we propose Motion TRansformer (MTR) framework that models motion prediction as the joint optimization of global intention localization and local movement refinement. Instead of using goal candidates, MTR incorporates spatial intention priors by adopting a small set of learnable motion query pairs. Each motion query pair takes charge of trajectory prediction and refinement for a specific motion mode, which stabilizes the training process and facilitates better multimodal predictions. Experiments show that MTR achieves state-of-the-art performance on both the marginal and joint motion prediction challenges, ranking 1st on the leaderboards of Waymo Open Motion Dataset. The source code is available at https://github.com/sshaoshuai/MTR.
Design Perspectives of Multitask Deep Learning Models and Applications

- In recent years, multi-task learning has turned out to be of great success in various applications. Though single model training has promised great results throughout these years, it ignores valuable information that might help us estimate a metric better. Under learning-related tasks, multi-task learning has been able to generalize the models even better. We try to enhance the feature mapping of the multi-tasking models by sharing features among related tasks and inductive transfer learning. Also, our interest is in learning the task relationships among various tasks for acquiring better benefits from multi-task learning. In this chapter, our objective is to visualize the existing multi-tasking models, compare their performances, the methods used to evaluate the performance of the multi-tasking models, discuss the problems faced during the design and implementation of these models in various domains, and the advantages and milestones achieved by them.
Reward Learning using Structural Motifs in Inverse Reinforcement Learning

The Inverse Reinforcement Learning (\textit{IRL}) problem has seen rapid evolution in the past few years, with important applications in domains like robotics, cognition, and health. In this work, we explore the inefficacy of current IRL methods in learning an agent's reward function from expert trajectories depicting long-horizon, complex sequential tasks. We hypothesize that imbuing IRL models with structural motifs capturing underlying tasks can enable and enhance their performance. Subsequently, we propose a novel IRL method, SMIRL, that first learns the (approximate) structure of a task as a finite-state-automaton (FSA), then uses the structural motif to solve the IRL problem. We test our model on both discrete grid world and high-dimensional continuous domain environments. We empirically show that our proposed approach successfully learns all four complex tasks, where two foundational IRL baselines fail. Our model also outperforms the baselines in sample efficiency on a simpler toy task. We further show promising test results in a modified continuous domain on tasks with compositional reward functions.
Lex2Sent: A bagging approach to unsupervised sentiment analysis

Unsupervised sentiment analysis is traditionally performed by counting those words in a text that are stored in a sentiment lexicon and then assigning a label depending on the proportion of positive and negative words registered. While these"counting"methods are considered to be beneficial as they rate a text deterministically, their classification rates decrease when the analyzed texts are short or the vocabulary differs from what the lexicon considers default. The model proposed in this paper, called Lex2Sent, is an unsupervised sentiment analysis method to improve the classification of sentiment lexicon methods. For this purpose, a Doc2Vec-model is trained to determine the distances between document embeddings and the embeddings of the positive and negative part of a sentiment lexicon. These distances are then evaluated for multiple executions of Doc2Vec on resampled documents and are averaged to perform the classification task. For three benchmark datasets considered in this paper, the proposed Lex2Sent outperforms every evaluated lexicon, including state-of-the-art lexica like VADER or the Opinion Lexicon in terms of classification rate.
Formal Conceptual Views in Neural Networks

Explaining neural network models is a challenging task that remains unsolved in its entirety to this day. This is especially true for high dimensional and complex data. With the present work, we introduce two notions for conceptual views of a neural network, specifically a many-valued and a symbolic view. Both provide novel analysis methods to enable a human AI analyst to grasp deeper insights into the knowledge that is captured by the neurons of a network. We test the conceptual expressivity of our novel views through different experiments on the ImageNet and Fruit-360 data sets. Furthermore, we show to which extent the views allow to quantify the conceptual similarity of different learning architectures. Finally, we demonstrate how conceptual views can be applied for abductive learning of human comprehensible rules from neurons. In summary, with our work, we contribute to the most relevant task of globally explaining neural networks models.
Exploring Low Rank Training of Deep Neural Networks

Training deep neural networks in low rank, i.e. with factorised layers, is of particular interest to the community: it offers efﬁciency over unfactorised training in terms of both memory con-sumption and training time. Prior work has focused on low rank approximations of pre-trained networks and training in low rank space with additional objectives, offering various ad hoc explana-tions for chosen practice. We analyse techniques that work well in practice, and through extensive ablations on models such as GPT2 we provide evidence falsifying common beliefs in the ﬁeld, hinting in the process at exciting research oppor-tunities that still need answering.
Uniqueness of the Stochastic Keller--Segel Model in One Dimension

In a recent paper (J. Diﬀerential Equations, 310 : 506–554, 2022), the authors proved the existence of martingale solutions to a stochastic version of the classical Patlak–Keller– Segel system in 1 dimension (1D), driven by time-homogeneous spatial Wiener processes. The current paper is a continuation and consists of two results about the stochastic Patlak–Keller– Segel system in 1D. First, we establish some additional regularity results of the solutions. The additional regularity is, e.g. important for its numerical modelling. Then, as a second result, we obtain the pathwise uniqueness of the solutions to the stochastic Patlak–Keller–Segel system in 1D. Finally, we conclude the paper with the existence of strong solution to this system in 1D.
Geometry-aware Two-scale PIFu Representation for Human Reconstruction

Although PIFu-based 3D human reconstruction methods are popular, the quality of recovered details is still unsatisfactory. In a sparse (e.g., 3 RGBD sensors) capture setting, the depth noise is typically amplified in the PIFu representation, resulting in flat facial surfaces and geometry-fallible bodies. In this paper, we propose a novel geometry-aware two-scale PIFu for 3D human reconstruction from sparse, noisy inputs. Our key idea is to exploit the complementary properties of depth denoising and 3D reconstruction, for learning a two-scale PIFu representation to reconstruct high-frequency facial details and consistent bodies separately. To this end, we first formulate depth denoising and 3D reconstruction as a multi-task learning problem. The depth denoising process enriches the local geometry information of the reconstruction features, while the reconstruction process enhances depth denoising with global topology information. We then propose to learn the two-scale PIFu representation using two MLPs based on the denoised depth and geometry-aware features. Extensive experiments demonstrate the effectiveness of our approach in reconstructing facial details and bodies of different poses and its superiority over state-of-the-art methods.
Slowly rotating α ′ -corrected black holes in four and higher dimensions

We consider the low-energy eﬀective action of string theory at order α ′ , including R 2 -corrections to the Einstein-Hilbert gravitational action and non-trivial dilaton coupling. By means of a convenient ﬁeld redeﬁnition, we manage to express the theory in a frame that enables us to solve its ﬁeld equations analytically and perturbatively in α ′ for a static spherically symmetric ansatz in an arbitrary number of dimensions. The set of solutions we obtain is compatible with asymptotically ﬂat geometries exhibiting a regular event horizon at which the dilaton is well-behaved. For the 4-dimensional case, we also derive the stationary black hole conﬁguration at ﬁrst order in α ′ and in the slowly rotating approximation. This yields string theory modiﬁcations to the Kerr geometry, including terms of the form a , a
Ver: View Discovery in the Wild

We present Ver1, a data discovery system that identifies project-join views over large repositories of tables that do not contain join path information, and even when input queries are inaccurate. Ver implements a reference architecture to solve both the technical (scale and search) and human (semantic ambiguity, navigating a large number of results) problems of view discovery. We demonstrate users find the view they want when using Ver with a user study and we demonstrate its performance with large-scale end-to-end experiments on real-world datasets containing tens of millions of join paths.
A SIMPLE GRAVITATIONAL LENS MODEL FOR COSMIC VOIDS

We present a simple gravitational lens model to illustrate the ease of using the embedded lensing theory when studying cosmic voids. It confirms the previously used repulsive lensing models for deep voids. We start by estimating magnitude fluctuations and weak-lensing shears of background sources lensed by large voids. We find that sources behind large (∼90 Mpc) and deep voids (density contrast about −0.9) can be magnified or demagnified with magnitude fluctuations of up to ∼0.05 mag and that the weak-lensing shear can be up to the ∼10−2 level in the outer regions of large voids. Smaller or shallower voids produce proportionally smaller effects. We investigate the “wiggling” of the primary cosmic microwave background (CMB) temperature anisotropies caused by intervening cosmic voids. The void-wiggling of primary CMB temperature gradients is of the opposite sign to that caused by galaxy clusters. Only extremely large and deep voids can produce wiggling amplitudes similar to galaxy clusters, ∼15 μK by a large void of radius ∼4° and central density contrast −0.9 at redshift 0.5 assuming a CMB background gradient of ∼10 μK arcmin−1. The dipole signal is spread over the entire void area, and not concentrated at the lens center as it is for clusters. Finally, we use our model to simulate CMB sky maps lensed by large cosmic voids. Our embedded theory can easily be applied to more complicated void models and used to study gravitational lensing of the CMB, to probe dark matter profiles, to reduce the lensing-induced systematics in supernova Hubble diagrams, and to study the integrated Sachs–Wolfe effect.
Core Lexicon and Contagious Words

We present the new empirical parameter $f_c$, the most probable usage frequency of a word in a language, computed via the distribution of documents over frequency $x$ of the word. This parameter allows for filtering the core lexicon of a language from the content words, which tend to be extremely frequent in some texts written in specific genres or by certain authors. Distributions of documents over frequencies for such words display long tails as $x>f_c$ representing a bunch of documents in which such words are used in abundance. Collections of such documents exhibit a percolation like phase transition as the coarse grain of frequency $\Delta f$ (flattening out the strongly irregular frequency data series) approaches the critical value $f_c$.
A Brief Introduction to Fluid Mechanics

Fluid Statics. Elementary Fluid DynamicsThe Bernoulli Equation. Fluid Kinematics. Finite Control Volume Analysis. Differential Analysis of Fluid Flow. Similitude, Dimensional Analysis, and Modeling. Viscous Flow in Pipes. Flow Over Immersed Bodies. Open-Channel Flow. Turbomachines. Appendices. Answers. Index.
OBSCURATION BY GAS AND DUST IN LUMINOUS QUASARS

We explore the connection between absorption by neutral gas and extinction by dust in mid-infrared (IR) selected luminous quasars. We use a sample of 33 quasars at redshifts 0.7 < z ≲ 3 in the 9 deg2 Boötes multiwavelength survey field that are selected using Spitzer Space Telescope Infrared Array Camera colors and are well-detected as luminous X-ray sources (with >150 counts) in Chandra observations. We divide the quasars into dust-obscured and unobscured samples based on their optical to mid-IR color, and measure the neutral hydrogen column density NH through fitting of the X-ray spectra. We find that all subsets of quasars have consistent power law photon indices Γ ≈ 1.9 that are uncorrelated with NH. We classify the quasars as gas-absorbed or gas-unabsorbed if NH > 1022 cm−2 or NH < 1022 cm−2, respectively. Of 24 dust-unobscured quasars in the sample, only one shows clear evidence for significant intrinsic NH, while 22 have column densities consistent with NH < 1022 cm−2. In contrast, of the nine dust-obscured quasars, six show evidence for intrinsic gas absorption, and three are consistent with NH < 1022 cm−2. We conclude that dust extinction in IR-selected quasars is strongly correlated with significant gas absorption as determined through X-ray spectral fitting. These results suggest that obscuring gas and dust in quasars are generally co-spatial, and confirm the reliability of simple mid-IR and optical photometric techniques for separating quasars based on obscuration.
Optimal Sampling of Random Fields for Topological Analysis

Algebraic topology is becoming an increasing important tool in applied mathematics. In particular, homology theory allows one to distinguish different topologies while being tractable to compute. An important application is the study of nodal domains for solutions to stochastic partial differential equations. These are the sets where the function value is greater than zero and less than zero. In order to compute the homology of the nodal sets computationally, we must discretize the domains. However, in this discretization process, we can make mistakes in the topology. Can we develop a method that will allow us to determine a proper discretization size a priori? One approach is to use an algorithm that is guaranteed to return the correct homology. The original algorithm has a few shortcomings. We will present these shortcomings and develop methods to overcome these issues. The other approach is to establish explicit probability bounds for the making the correct homology. This is an a priori approach that will returns the probability for a fixed discretization size and also determines the optimal location of the sampled points.
Size- and environment-driven seedling survival and growth are mediated by leaf functional traits

Ecologists usually find that plant demography (e.g. survival and growth) changes along with plant size and environmental gradients, which suggests the effects of ontogeny-related processes and abiotic filtering. However, the role of functional traits underlying the size– and environment–demography relationships is usually overlooked. By measuring individual-level leaf traits of more than 2700 seedlings in a temperate forest, we evaluated how seedling traits mediated the size– and environment–demography relationships. We found leaves were larger for taller seedlings; leaf economics traits were more conservative in taller seedlings and under high-light and low-elevation conditions. Structural equation modelling showed that a higher survival probability for taller seedlings was indirectly driven by their larger leaf area. Although taller seedlings had lower growth rates, larger and more resource-conservative leaves could promote the growth of these tall seedlings. Environmental variables did not influence seedling survival and growth directly but did influence growth indirectly by mediating trait variation. Finally, species-specific variation in traits along with size and environments was associated with the species-specific variation in seedling survival and growth. Our study suggests that not only plant ontogeny- and environment-related ecological processes, but functional traits are also important intermediary agents underlying plant size– and environment–demography relationships.
Developmental bias in the evolution and plasticity of beetle horn shape

The degree to which developmental systems bias the phenotypic effects of environmental and genetic variation, and how these biases affect evolution, is subject to much debate. Here, we assess whether developmental variability in beetle horn shape aligns with the phenotypic effects of plasticity and evolutionary divergence, yielding three salient results. First, we find that most pathways previously shown to regulate horn length also affect shape. Second, we find that the phenotypic effects of manipulating divergent developmental pathways are correlated with each other as well as multivariate fluctuating asymmetry—a measure of developmental variability. Third, these effects further aligned with thermal plasticity, population differences and macroevolutionary divergence between sister taxa and more distantly related species. Collectively, our results support the hypothesis that changes in horn shape—whether brought about by environmentally plastic responses, functional manipulations or evolutionary divergences—converge along ‘developmental lines of least resistance’, i.e. are biased by the developmental system underpinning horn shape.
Fossil basicranium clarifies the origin of the avian central nervous system and inner ear

Among terrestrial vertebrates, only crown birds (Neornithes) rival mammals in terms of relative brain size and behavioural complexity. Relatedly, the anatomy of the avian central nervous system and associated sensory structures, such as the vestibular system of the inner ear, are highly modified with respect to those of other extant reptile lineages. However, a dearth of three-dimensional Mesozoic fossils has limited our knowledge of the origins of the distinctive endocranial structures of crown birds. Traits such as an expanded, flexed brain, a ventral connection between the brain and spinal column, and a modified vestibular system have been regarded as exclusive to Neornithes. Here, we demonstrate all of these ‘advanced’ traits in an undistorted braincase from an Upper Cretaceous enantiornithine bonebed in southeastern Brazil. Our discovery suggests that these crown bird-like endocranial traits may have originated prior to the split between Enantiornithes and the more crownward portion of avian phylogeny over 140 Ma, while coexisting with a remarkably plesiomorphic cranial base and posterior palate region. Altogether, our results support the interpretation that the distinctive endocranial morphologies of crown birds and their Mesozoic relatives are affected by complex trade-offs between spatial constraints during development.
Forecasting the publication and citation outcomes of COVID-19 preprints

Many publications on COVID-19 were released on preprint servers such as medRxiv and bioRxiv. It is unknown how reliable these preprints are, and which ones will eventually be published in scientific journals. In this study, we use crowdsourced human forecasts to predict publication outcomes and future citation counts for a sample of 400 preprints with high Altmetric score. Most of these preprints were published within 1 year of upload on a preprint server (70%), with a considerable fraction (45%) appearing in a high-impact journal with a journal impact factor of at least 10. On average, the preprints received 162 citations within the first year. We found that forecasters can predict if preprints will be published after 1 year and if the publishing journal has high impact. Forecasts are also informative with respect to Google Scholar citations within 1 year of upload on a preprint server. For both types of assessment, we found statistically significant positive correlations between forecasts and observed outcomes. While the forecasts can help to provide a preliminary assessment of preprints at a faster pace than traditional peer-review, it remains to be investigated if such an assessment is suited to identify methodological problems in preprints.
Contrastive Representation Learning for Expression Recognition from Masked Face Images

With the worldwide spread of COVID-19, people are trying different ways to prevent the spread of the virus. One of the most useful and popular ways is wearing a face mask. Most people wear a face mask when they go out, which makes facial expression recognition become harder. Thus, how to improve the performance of the facial expression recognition model on masked faces is becoming an important issue. However, there is no public dataset that includes facial expressions with masks. Thus, we built two datasets which are a real-world masked facial expression database (VIP-DB) and a man-made masked facial expression database (M-RAF-DB). To reduce the influence of masks, we utilize contrastive representation learning and propose a two-branches network. We study the influence of contrastive learning on our two datasets. Results show that using contrastive representation learning improves the performance of expression recognition from masked face images.
Hardness Characterisations and Size-width Lower Bounds for QBF Resolution

We provide a tight characterisation of proof size in resolution for quantified Boolean formulas (QBF) via circuit complexity. Such a characterisation was previously obtained for a hierarchy of QBF Frege systems [16], but leaving open the most important case of QBF resolution. Different from the Frege case, our characterisation uses a new version of decision lists as its circuit model, which is stronger than the CNFs the system works with. Our decision list model is well suited to compute countermodels for QBFs. Our characterisation works for both Q-Resolution and QU-Resolution. Using our characterisation, we obtain a size-width relation for QBF resolution in the spirit of the celebrated result for propositional resolution [4]. However, our result is not just a replication of the propositional relation—intriguingly ruled out for QBF in previous research [12]—but shows a different dependence between size, width, and quantifier complexity. An essential ingredient is an improved relation between the size and width of term decision lists; this may be of independent interest. We demonstrate that our new technique elegantly reproves known QBF hardness results and unifies previous lower-bound techniques in the QBF domain.