Miracle-dz/UniFAR-FTUs huggingface dataset:
Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving

Aiming towards a holistic understanding of multiple downstream tasks simultaneously, there is a need for extracting features with better transferability. Though many latest self-supervised pre-training methods have achieved impressive performance on various vision tasks under the prevailing pretrain-finetune paradigm, their generalization capacity to multi-task learning scenarios is yet to be explored. In this paper, we extensively investigate the transfer performance of various types of self-supervised methods, e.g., MoCo and SimCLR, on three downstream tasks, including semantic segmentation, drivable area segmentation, and traffic object detection, on the large-scale driving dataset BDD100K. We surprisingly find that their performances are sub-optimal or even lag far behind the single-task baseline, which may be due to the distinctions of training objectives and architectural design lied in the pretrain-finetune paradigm. To overcome this dilemma as well as avoid redesigning the resource-intensive pre-training stage, we propose a simple yet effective pretrain-adapt-finetune paradigm for general multi-task training, where the off-the-shelf pretrained models can be effectively adapted without increasing the training overhead. During the adapt stage, we utilize learnable multi-scale adapters to dynamically adjust the pretrained model weights supervised by multi-task objectives while leaving the pretrained knowledge untouched. Furthermore, we regard the vision-language pre-training model CLIP as a strong complement to the pretrain-adapt-finetune paradigm and propose a novel adapter named LV-Adapter, which incorporates language priors in the multi-task model via task-specific prompting and alignment between visual and textual features.
Deep Metric Learning with Chance Constraints

Deep metric learning (DML) aims to minimize empirical expected loss of the pairwise intra-/inter- class proximity violations in the embedding space. We relate DML to feasibility problem of finite chance constraints. We show that minimizer of proxy-based DML satisfies certain chance constraints, and that the worst case generalization performance of the proxy-based methods can be characterized by the radius of the smallest ball around a class proxy to cover the entire domain of the corresponding class samples, suggesting multiple proxies per class helps performance. To provide a scalable algorithm as well as exploiting more proxies, we consider the chance constraints implied by the minimizers of proxy-based DML instances and reformulate DML as finding a feasible point in intersection of such constraints, resulting in a problem to be approximately solved by iterative projections. Simply put, we repeatedly train a regularized proxy-based loss and re-initialize the proxies with the embeddings of the deliberately selected new samples. We applied our method with 4 well-accepted DML losses and show the effectiveness with extensive evaluations on 4 popular DML benchmarks. Code is available at: https://github.com/yetigurbuz/ccp-dml
Efficient and Consistent Bundle Adjustment on Lidar Point Clouds

Simultaneous determination of sensor poses and scene geometry is a fundamental problem for robot vision that is often achieved by Bundle Adjustment (BA). This article presents an efficient and consistent bundle adjustment method for light detection and ranging (lidar) sensors. The method employs edge and plane features to represent the scene geometry, and directly minimizes the natural Euclidean distance from each raw point to the respective geometry feature. A nice property of this formulation is that the geometry features can be analytically solved, drastically reducing the dimension of the numerical optimization. To represent and solve the resultant optimization problem more efficiently, this paper then adopts and formalizes the concept of point cluster, which encodes all raw points associated to the same feature by a compact set of parameters, the point cluster coordinates. We derive the closed-form derivatives, up to the second order, of the BA optimization based on the point cluster coordinates and show their theoretical properties such as the null spaces and sparsity. Based on these theoretical results, this paper develops an efficient second-order BA solver. Besides estimating the lidar poses, the solver also exploits the second order information to estimate the pose uncertainty caused by measurement noises, leading to consistent estimates of lidar poses. Moreover, thanks to the use of point cluster, the developed solver fundamentally avoids the enumeration of each raw point in all steps of the optimization: cost evaluation, derivatives evaluation and uncertainty evaluation. The implementation of our method is open sourced to benefit the robotics community.
Geometric Brownian information engine: Essentials for the best performance.

We investigate a geometric Brownian information engine (GBIE) in the presence of an error-free feedback controller that transforms the information gathered on the state of Brownian particles entrapped in monolobal geometric confinement into extractable work. Outcomes of the information engine depend on the reference measurement distance x_{m}, the feedback site x_{f}, and the transverse force G. We determine the benchmarks for utilizing the available information in an output work and the optimum operating requisites for best achievable work. Transverse bias force (G) tunes the entropic contribution in the effective potential and hence the standard deviation (σ) of the equilibrium marginal probability distribution. We recognize that the amount of extractable work reaches a global maximum when x_{f}=2x_{m} with x_{m}∼0. 6σ, irrespective of the extent of the entropic limitation. Because of the higher loss of information during the relaxation process, the best achievable work of a GBIE is lower in an entropic system. The feedback regulation also bears the unidirectional passage of particles. The average displacement increases with growing entropic control and is maximum when x_{m}∼0. 81σ. Finally, we explore the efficacy of the information engine, a quantity that regulates the efficiency in utilizing the information acquired. With x_{f}=2x_{m}, the maximum efficacy reduces with increasing entropic control and shows a crossover from 2 to 11/9. We discover that the condition for the best efficacy depends only on the confinement lengthscale along the feedback direction. The broader marginal probability distribution accredits the increased average displacement in a cycle and the lower efficacy in an entropy-dominated system.
Finitude of physical measures for random maps

For random compositions of independent and identically distributed measurable maps on a Polish space, we study the existence and finitude of absolutely continuous ergodic stationary probability measures (which are, in particular, physical measures) whose basins of attraction cover the whole space almost everywhere. We characterize and hierarchize such random maps in terms of their associated Markov operators, as well as show the difference between classes in the hierarchy by plenty of examples, including additive noise, multiplicative noise, and iterated function systems. We also provide sufficient practical conditions for a random map to belong to these classes. For instance, we establish that any continuous random map on a compact Riemannian manifold with absolutely continuous transition probability has finitely many physical measures whose basins of attraction cover Lebesgue almost all the manifold.
A computational model of learning flexible navigation in a maze by layout-conforming replay of place cells

Recent experimental observations have shown that the reactivation of hippocampal place cells (PC) during sleep or immobility depicts trajectories that can go around barriers and can flexibly adapt to a changing maze layout. Such layout-conforming replay sheds a light on how the activity of place cells supports the learning of flexible navigation in a dynamically changing maze. However, existing computational models of replay fall short of generating layout-conforming replay, restricting their usage to simple environments, like linear tracks or open fields. In this paper, we propose a computational model that generates layout-conforming replay and explains how such replay drives the learning of flexible navigation in a maze. First, we propose a Hebbian-like rule to learn the inter-PC synaptic strength during exploring a maze. Then we use a continuous attractor network (CAN) with feedback inhibition to model the interaction among place cells and hippocampal interneurons. The activity bump of place cells drifts along a path in the maze, which models layout-conforming replay. During replay in rest, the synaptic strengths from place cells to striatal medium spiny neurons (MSN) are learned by a novel dopamine-modulated three-factor rule to store place-reward associations. During goal-directed navigation, the CAN periodically generates replay trajectories from the animal’s location for path planning, and the trajectory leading to a maximal MSN activity is followed by the animal. We have implemented our model into a high-fidelity virtual rat in the MuJoCo physics simulator. Extensive experiments have demonstrated that its superior flexibility during navigation in a maze is due to a continuous re-learning of inter-PC and PC-MSN synaptic strength.
Safety Index Synthesis via Sum-of-Squares Programming

Control systems often need to satisfy strict safety requirements. Safety index provides a handy way to evaluate the safety level of the system and derive the resulting safe control policies. However, designing safety index functions under control limits is difficult and requires a great amount of expert knowledge. This paper proposes a framework for synthesizing the safety index for general control systems using sum-of-squares programming. Our approach is to show that ensuring the non-emptiness of safe control on the safe set boundary is equivalent to a local manifold positiveness problem. We then prove that this problem is equivalent to sum-of-squares programming via the Positivstellensatz of algebraic geometry. We validate the proposed method on robot arms with different degrees of freedom and ground vehicles. The results show that the synthesized safety index guarantees safety and our method is effective even in high-dimensional robot systems.
Strong Dzyaloshinskii-Moriya interaction in monolayer 
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msub><mml:mi>CrI</mml:mi><mml:mn>3</mml:mn></mml:msub></mml:math>
 on metal substrates

Dzyaloshinskii-Moriya interaction (DMI) is the primary mechanism for realizing real-space chiral spin textures, which are regarded as key components for the next-generation spintronics. However, DMI arises from a perturbation term of the spin-orbit interaction and is usually weak in pristine magnetic semiconductors. To date, large DMI and the resulting skyrmions are only realized in a few materials under stringent conditions. Using first-principles calculations, we demonstrate that significant DMI occurs between nearest-neighbor Cr atoms in two-dimensional (2D) magnetic semiconductor CrI$_3$ on Au or Cu substrates. This exceptionally strong DMI is generated by the interfacial charge transfer and weak chemical interactions between chromium halides and metal substrates, which break the spatial inversion symmetry. These findings highlight the significance of substrate effects in 2D magnets and expand the inventory of feasible materials with strong DMI.
LMBAO: A Landmark Map for Bundle Adjustment Odometry in LiDAR SLAM

Existing LiDAR odometry strategies match a new scan iteratively with previous fixed-pose scans, gradually accumulating errors. Furthermore, as an effective joint optimization mechanism, bundle adjustment (BA) cannot be directly introduced into odometry due to the intensive computation of global landmarks. Therefore, this paper designs a landmark map for bundle adjustment odometry (LMBAO) in LiDAR SLAM. First, an active landmark maintenance strategy is developed to obtain a local map of limited size that enables real-time BA. Specifically, this paper keeps entire stable landmarks on the map instead of just their feature points in the sliding window and timely deletes inactive landmarks. Next, unlike visual marginalization to approximate the Gaussian distribution, and a direct and efficient marginalization strategy is performed to retain the scans outside the window to greatly simplifying the computation. Experiments show the effectiveness of LMBAO in outdoor driving.
The Impact of Binaries on the Dynamical Mass Estimate of Dwarf Galaxies

Binary stars are recognized to be important in driving the dynamical evolution of stellar systems and also in determining some of their observational features. In this study, we explore the role that binary stars have in modulating the estimates of the velocity dispersion of stellar systems. To this aim, we developed a tool that allows investigation of the dependence of synthetic velocity dispersion on a number of crucial quantities characterizing the binary content: binary fraction and the distributions of their mass ratio, eccentricity, and semimajor axis. As an application, we evaluate the impact that binary stars have on the estimation of the dynamical mass of dwarf spheroidal and ultrafaint dwarf galaxies, finding that it can be particularly relevant, especially for low-mass and low-density systems. These results bear profound implications for the interpretation of the measured velocity dispersion in such systems, since it weakens or relieves the claim for the need of large amounts of dark matter.
Three-dimensional non-kinematic simulation of the post-emergence evolution of bipolar magnetic regions and the Babcock-Leighton dynamo of the Sun

The Babcock-Leighton (BL) flux-transport model is a widely-accepted dynamo model of the Sun. This dynamo model has been extensively studied in a two-dimensional (2D) mean-field framework in both kinematic and non-kinematic regimes. Recent three-dimensional (3D) models have been restricted to the kinematic regime. In these models, the surface poloidal flux is produced by the emergence of bipolar magnetic regions (BMRs) that are tilted according to Joy's law. We investigate the prescription for emergence of a BMR in 3D non-kinematic simulations. We also report initial results of cyclic BL dynamo simulation. We extend a conventional 2D mean-field model of the BL flux-transport dynamo into 3D non-kinematic regime. The large-scale mean flows are driven by the parameterized $\Lambda$-effect in this model. For the induction equation, we use a BL source term by which the surface BMRs are produced in response to the dynamo-generated toroidal field inside the convection zone. We find that, in the 3D non-kinematic regime, the tilt angle of a newly-emerged BMR is very sensitive to the prescription for the subsurface structure of the BMR. Anti-Joy tilt angles are found unless the BMR is deeply embedded in the convection zone. We also find that the leading spot tends to become stronger than the following spot. The anti-Joy's law trend and the morphological asymmetry of the BMRs can be explained by the Coriolis force acting on the Lorentz-force-driven flows. Furthermore, we demonstrate that the solar-like magnetic cycles can be successfully obtained if the Joy's law is explicitly given in the BL $\alpha$-effect. In these cyclic dynamo simulation, a strong Lorentz force feedback leads to cycle modulations in the differential rotation and meridional circulation. The non-axisymmetric components of the flows are found to exist as inertial modes such as the equatorial Rossby modes.
Progressive tearing and cutting of soft-bodies in high-performance virtual reality

We present an algorithm that allows a user within a virtual environment to perform real-time unconstrained cuts or consecutive tears, i.e., progressive, continuous fractures on a deformable rigged and soft-body mesh model in high-performance 10ms. In order to recreate realistic results for different physically-principled materials such as sponges, hard or soft tissues, we incorporate a novel soft-body deformation, via a particle system layered on-top of a linear-blend skinning model. Our framework allows the simulation of realistic, surgical-grade cuts and continuous tears, especially valuable in the context of medical VR training. In order to achieve high performance in VR, our algorithms are based on Euclidean geometric predicates on the rigged mesh, without requiring any specific model pre-processing. The contribution of this work lies on the fact that current frameworks supporting similar kinds of model tearing, either do not operate in high-performance real-time or only apply to predefined tears. The framework presented allows the user to freely cut or tear a 3D mesh model in a consecutive way, under 10ms, while preserving its soft-body behaviour and/or allowing further animation.
DytanVO: Joint Refinement of Visual Odometry and Motion Segmentation in Dynamic Environments

Learning-based visual odometry (VO) algorithms achieve remarkable performance on common static scenes, benefiting from high-capacity models and massive annotated data, but tend to fail in dynamic, populated environments. Semantic segmentation is largely used to discard dynamic associations before estimating camera motions but at the cost of discarding static features and is hard to scale up to unseen categories. In this paper, we leverage the mutual dependence between camera ego-motion and motion segmentation and show that both can be jointly refined in a single learning-based framework. In particular, we present DytanVO, the first supervised learning-based VO method that deals with dynamic environments. It takes two consecutive monocular frames in real-time and predicts camera ego-motion in an iterative fashion. Our method achieves an average improvement of 27.7% in ATE over state-of-the-art VO solutions in real-world dynamic environments, and even performs competitively among dynamic visual SLAM systems which optimize the trajectory on the backend. Experiments on plentiful unseen environments also demonstrate our method's generalizability.
Spectrally stable nitrogen-vacancy centers in diamond formed by carbon implantation into thin microstructures

The nitrogen-vacancy center (NV) in diamond, with its exceptional spin coherence and convenience in optical spin initialization and readout, is increasingly used both as a quantum sensor and as a building block for quantum networks. Employing photonic structures for maximizing the photon collection efficiency in these applications typically leads to broadened optical linewidths for the emitters, which are commonly created via nitrogen ion implantation. With studies showing that only native nitrogen atoms contribute to optically coherent NVs, a natural conclusion is to either avoid implantation completely or substitute nitrogen implantation by an alternative approach to vacancy creation. Here, we demonstrate that implantation of carbon ions yields a comparable density of NVs as implantation of nitrogen ions and that it results in NV populations with narrow optical linewidths and low charge-noise levels even in thin diamond microstructures. We measure a median NV linewidth of 150 MHz for structures thinner than 5 μm, with no trend of increasing linewidths down to the thinnest measured structure of 1.9 μm. We propose a modified NV creation procedure in which the implantation is carried out after instead of before the diamond fabrication processes and confirm our results in multiple samples implanted with different ion energies and fluences.
BareSkinNet: De‐makeup and De‐lighting via 3D Face Reconstruction

We propose BareSkinNet, a novel method that simultaneously removes makeup and lighting influences from the face image. Our method leverages a 3D morphable model and does not require a reference clean face image or a specified light condition. By combining the process of 3D face reconstruction, we can easily obtain 3D geometry and coarse 3D textures. Using this information, we can infer normalized 3D face texture maps (diffuse, normal, roughness, and specular) by an image‐translation network. Consequently, reconstructed 3D face textures without undesirable information will significantly benefit subsequent processes, such as re‐lighting or re‐makeup. In experiments, we show that BareSkinNet outperforms state‐of‐the‐art makeup removal methods. In addition, our method is remarkably helpful in removing makeup to generate consistent high‐fidelity texture maps, which makes it extendable to many realistic face generation applications. It can also automatically build graphic assets of face makeup images before and after with corresponding 3D data. This will assist artists in accelerating their work, such as 3D makeup avatar creation.
A fast two-level Strang splitting method for multi-dimensional spatial fractional Allen-Cahn equations with discrete maximum principle

In this paper, we study the numerical solutions of the multi-dimensional spatial fractional Allen-Cahn equations. After semi-discretization for the spatial fractional Riesz derivative, a system of nonlinear ordinary differential equations with Toeplitz structure is obtained. For the sake of reducing the computational complexity, a two-level Strang splitting method is proposed, where the Toeplitz matrix in the system is split into the sum of a circulant matrix and a skew-circulant matrix. Therefore, the proposed method can be quickly implemented by the fast Fourier transform, substituting to calculate the expensive Toeplitz matrix exponential. Theoretically, the discrete maximum principle of our method is unconditionally preserved. Moreover, the analysis of error in the infinite norm with second-order accuracy is conducted in both time and space. Finally, numerical tests are given to corroborate our theoretical conclusions and the efficiency of the proposed method.
A Type-Directed, Dictionary-Passing Translation of Featherweight Generic Go

Featherweight Generic Go (FGG) is a minimal core calculus modeling the essential features of the programming language Go. It includes support for overloaded methods, interface types, structural subtyping and generics. The most straightforward semantic description of the dynamic behavior of FGG programs is to resolve method calls based on runtime type information of the receiver. This article shows a different approach by defining a type-directed translation from FGG to an untyped lambda-calculus. The translation of an FGG program provides evidence for the availability of methods as additional dictionary parameters, similar to the dictionary-passing approach known from Haskell type classes. Then, method calls can be resolved by a simple lookup of the method definition in the dictionary. Every program in the image of the translation has the same dynamic semantics as its source FGG program. The proof of this result is based on a syntactic, step-indexed logical relation. The stepindex ensures a well-founded definition of the relation in the presence of recursive interface types and recursive methods.