Miracle-dz/UniFAR-FTUs huggingface dataset:
Long-Tailed 3D Detection via 2D Late Fusion

Autonomous vehicles (AVs) must accurately detect objects from both common and rare classes for safe navigation, motivating the problem of Long-Tailed 3D Object Detection (LT3D). Contemporary LiDAR-based 3D detectors perform poorly on rare classes (e.g., CenterPoint only achieves 5.1 AP on stroller) as it is difficult to recognize objects from sparse LiDAR points alone. RGB images provide visual evidence to help resolve such ambiguities, motivating the study of RGB-LiDAR fusion. In this paper, we delve into a simple late-fusion framework that ensembles independently trained RGB and LiDAR detectors. Unlike recent end-to-end methods which require paired multi-modal training data, our late-fusion approach can easily leverage large-scale uni-modal datasets, significantly improving rare class detection. In particular, we examine three critical components in this late-fusion framework from first principles, including whether to train 2D or 3D RGB detectors, whether to match RGB and LiDAR detections in 3D or the projected 2D image plane, and how to fuse matched detections.Extensive experiments reveal that 2D RGB detectors achieve better recognition accuracy than 3D RGB detectors, matching on the 2D image plane mitigates depth estimation errors, and fusing scores probabilistically with calibration leads to state-of-the-art LT3D performance. Our late-fusion approach achieves 51.4 mAP on the established nuScenes LT3D benchmark, improving over prior work by 5.9 mAP.
Observation of significant flavor-SU(3) breaking in the kaon wave function at $12~{\rm GeV}^2

We present cross sections for the reaction $e^+e^-\to K_S^0K_L^0$ at center-of-mass energies ranging from 3.51 GeV to 4.95 GeV using data samples collected in the BESIII experiment, corresponding to a total integrated luminosity of 26.5 fb$^{-1}$. The ratio of neutral-to-charged kaon form factors at large momentum transfers ($12~{\rm GeV}^2<Q^2<25~{\rm GeV}^2$) is determined to be $0.21\pm 0.01$, which indicates a small but significant effect of flavor-SU(3) breaking in the kaon wave function, and consequently excludes the possibility that flavor-SU(3) breaking is the primary reason for the strong experimental violation of the pQCD prediction $|F(\pi^{\pm})|/|F(K^{\pm})|=f^2_{\pi}/f^2_{K}$, where $F(\pi^{\pm})$ and $F(K^{\pm})$ are the form factors, and $f_{\pi}$ and $f_{K}$ are the decay constants of charged pions and kaons, respectively. We also observe a significant signal for the charmless decay $\psi(3770)\to K_S^0K_L^0$ for the first time. Within a $1\sigma$ contour of the likelihood value, the the branching fraction for $\psi(3770)\to K_S^0K_L^0$ is determined to be ${\cal B}=(2.63_{-1.59}^{+1.40})\times 10^{-5}$, and the relative phase between the continuum and $\psi(3770)$ amplitudes is $\phi=(-0.39_{-0.10}^{+0.05})\pi$. The branching fraction is in good agreement with the $\mathcal{S}$- and $\mathcal{D}$-wave charmonia mixing scheme proposed in the interpretation of the"$\rho\pi$ puzzle"between $J/\psi$ and $\psi(3686)$ decays.
Data-based Model Identification of the Hypothalamus-Pituitary-Thyroid Complex

The thyroid gland, in conjunction with the pituitary and the hypothalamus, forms a regulated system due to their mutual influence through released hormones. The equilibrium point of this system, commonly referred to as the"set point", is individually determined. This means that determining the correct amount of medication to be administered to patients with hypothyroidism requires several treatment appointments creating an extended treatment process. Because the dynamics of the system have not yet been fully explored, mathematical models are needed to simulate the mutual influence of the respective hormones as well as their course over time. These models enable a deeper understanding of the functionality in the context of data measurements. Therefore, two existing time-dependent mathematical models are used and further analyzed to replicate this overall influence of disparate systems. Both are based on a system of two differential equations modelling the interacting hormones. The parameters of the two models are identified according to different calibration approaches by means of patient data collected in a retrospective study in collaboration with the Medical University of Vienna. The hormonal course in the time domain as well as equilibrium curves including the set-point are then simulated and analyzed with respect to the normalized mean squared error. These calibrated systems allow a more profound insight into the functionality of the formed complex.
Decoupled Knowledge with Ensemble Learning for Online Distillation

Offline distillation is a two-stage pipeline that requires expensive resources to train a teacher network and then distill the knowledge to a student for deployment. Online knowledge distillation, on the other hand, is a one-stage strategy that alleviates the requirement with mutual learning and collaborative learning. Recent peer collaborative learning (PCL) integrates online ensemble, collaboration of base networks and temporal mean teacher to construct effective knowledge. However, the model collapses occasionally in PCL due to high homogenization between the student and the teacher. In this paper, the cause of the high homogenization is analyzed and the solution is presented. A decoupled knowledge for online knowledge distillation is generated by an independent teacher, separate from the student. Such design can increase the diversity between the networks and reduce the possibility of model collapse. To obtain early decoupled knowledge, an initialization scheme for the teacher is devised, and a 2D geometry-based analysis experiment is conducted under ideal conditions to showcase the effectiveness of this scheme. Moreover, to improve the teacher's supervisory resilience, a decaying ensemble scheme is devised. It assembles the knowledge of the teacher to which a dynamic weight which is large at the start of the training and gradually decreases with the training process is assigned. The assembled knowledge serves as a strong teacher during the early training and the decreased-weight-assembled knowledge can eliminate the distribution deviation under the potentially overfitted teacher's supervision. A Monte Carlo-based simulation is conducted to evaluate the convergence. Extensive experiments on CIFAR-10, CIFAR-100 and TinyImageNet show the superiority of our method. Ablation studies and further analysis demonstrate the effectiveness.
The Pros and Cons of Adversarial Robustness

Robustness is widely regarded as a fundamental problem in the analysis of machine learning (ML) models. Most often robustness equates with deciding the non-existence of adversarial examples, where adversarial examples denote situations where small changes on some inputs cause a change in the prediction. The perceived importance of ML model robustness explains the continued progress observed for most of the last decade. Whereas robustness is often assessed locally, i.e. given some target point in feature space, robustness can also be defined globally, i.e. where any point in feature space can be considered. The importance of ML model robustness is illustrated for example by the existence of competitions evaluating the progress of robustness tools, namely in the case of neural networks (NNs) but also by efforts towards robustness certification. More recently, robustness tools have also been used for computing rigorous explanations of ML models. In contrast with the observed successes of robustness, this paper uncovers some limitations with existing definitions of robustness, both global and local, but also with efforts towards robustness certification. The paper also investigates uses of adversarial examples besides those related with robustness.
Benchmarks for Physical Reasoning AI

Physical reasoning is a crucial aspect in the development of general AI systems, given that human learning starts with interacting with the physical world before progressing to more complex concepts. Although researchers have studied and assessed the physical reasoning of AI approaches through various specific benchmarks, there is no comprehensive approach to evaluating and measuring progress. Therefore, we aim to offer an overview of existing benchmarks and their solution approaches and propose a unified perspective for measuring the physical reasoning capacity of AI systems. We select benchmarks that are designed to test algorithmic performance in physical reasoning tasks. While each of the selected benchmarks poses a unique challenge, their ensemble provides a comprehensive proving ground for an AI generalist agent with a measurable skill level for various physical reasoning concepts. This gives an advantage to such an ensemble of benchmarks over other holistic benchmarks that aim to simulate the real world by intertwining its complexity and many concepts. We group the presented set of physical reasoning benchmarks into subcategories so that more narrow generalist AI agents can be tested first on these groups.
Passive Photonic Phase Gate via a Two-Level Emitter and Few Cavities

We show that a simple system consisting of a two-level emitter and few cavities could realize a high-fidelity deterministic controlled-$\pi$-phase gate for traveling single-photon qubits. The gate relies on the optimal setting of the coupling rates among the emitter and cavities and the use of photon wavepackets with an appropriate temporal shape, which could simply the gate operation to a process of complete absorption and re-emission of the wavepackets. consequently, it is free of wavepacket distortions, circumventing the long-standing challenge associated with the use of nonlinear media as a phase gate. Undergoing the process of absorption and re-emission, the two-level emitter enables a nonlinear $\pi$ phase shift for the two-photon wavepacket. The gate fidelity could reach over 99% with only four cavities. The proposed gate is passive and its architecture is compatible with integrated photonic platforms and in line with recent developments in quantum photonics.
Information Type Classification with Contrastive Task-Specialized Sentence Encoders

User-generated information content has become an important information source in crisis situations. However, classification models suffer from noise and event-related biases which still poses a challenging task and requires sophisticated task-adaptation. To address these challenges, we propose the use of contrastive task-specialized sentence encoders for downstream classification. We apply the task-specialization on the CrisisLex, HumAID, and TrecIS information type classification tasks and show performance gains w.r. t. F1-score. Furthermore, we analyse the cross-corpus and cross-lingual capabilities for two German event relevancy classification datasets.
Factor Risk Budgeting and Beyond

Portfolio optimization methods have evolved significantly since Markowitz introduced the mean-variance framework in 1952. While the theoretical appeal of this approach is undeniable, its practical implementation poses important challenges, primarily revolving around the intricate task of estimating expected returns. As a result, practitioners and scholars have explored alternative methods that prioritize risk management and diversification. One such approach is Risk Budgeting, where portfolio risk is allocated among assets according to predefined risk budgets. The effectiveness of Risk Budgeting in achieving true diversification can, however, be questioned, given that asset returns are often influenced by a small number of risk factors. From this perspective, one question arises: is it possible to allocate risk at the factor level using the Risk Budgeting approach? This paper introduces a comprehensive framework to address this question by introducing risk measures directly associated with risk factor exposures and demonstrating the desirable mathematical properties of these risk measures, making them suitable for optimization. We also propose a framework to find the portfolio that effectively balances the risk contributions from both assets and factors. Leveraging standard stochastic algorithms, our framework enables the use of a wide range of risk measures.
Robust Active Measuring under Model Uncertainty

Partial observability and uncertainty are common problems in sequential decision-making that particularly impede the use of formal models such as Markov decision processes (MDPs). However, in practice, agents may be able to employ costly sensors to measure their environment and resolve partial observability by gathering information. Moreover, imprecise transition functions can capture model uncertainty. We combine these concepts and extend MDPs to robust active-measuring MDPs (RAM-MDPs). We present an active-measure heuristic to solve RAM-MDPs efficiently and show that model uncertainty can, counterintuitively, let agents take fewer measurements. We propose a method to counteract this behavior while only incurring a bounded additional cost. We empirically compare our methods to several baselines and show their superior scalability and performance.
3S-TSE: Efficient Three-Stage Target Speaker Extraction for Real-Time and Low-Resource Applications

Target speaker extraction (TSE) aims to isolate a specific voice from multiple mixed speakers relying on a registerd sample. Since voiceprint features usually vary greatly, current end-to-end neural networks require large model parameters which are computational intensive and impractical for real-time applications, espetially on resource-constrained platforms. In this paper, we address the TSE task using microphone array and introduce a novel three-stage solution that systematically decouples the process: First, a neural network is trained to estimate the direction of the target speaker. Second, with the direction determined, the Generalized Sidelobe Canceller (GSC) is used to extract the target speech. Third, an Inplace Convolutional Recurrent Neural Network (ICRN) acts as a denoising post-processor, refining the GSC output to yield the final separated speech. Our approach delivers superior performance while drastically reducing computational load, setting a new standard for efficient real-time target speaker extraction.
Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective

Generative Adversarial Imitation Learning (GAIL) stands as a cornerstone approach in imitation learning. This paper investigates the gradient explosion in two types of GAIL: GAIL with deterministic policy (DE-GAIL) and GAIL with stochastic policy (ST-GAIL). We begin with the observation that the training can be highly unstable for DE-GAIL at the beginning of the training phase and end up divergence. Conversely, the ST-GAIL training trajectory remains consistent, reliably converging. To shed light on these disparities, we provide an explanation from a theoretical perspective. By establishing a probabilistic lower bound for GAIL, we demonstrate that gradient explosion is an inevitable outcome for DE-GAIL due to occasionally large expert-imitator policy disparity, whereas ST-GAIL does not have the issue with it. To substantiate our assertion, we illustrate how modifications in the reward function can mitigate the gradient explosion challenge. Finally, we propose CREDO, a simple yet effective strategy that clips the reward function during the training phase, allowing the GAIL to enjoy high data efficiency and stable trainability.
An appointment with Reproducing Kernel Hilbert Space generated by Generalized Gaussian RBF as L2-measure

Gaussian Radial Basis Function (RBF) Kernels are the most-often-employed kernels in artificial intelligence and machine learning routines for providing optimally-best results in contrast to their respective counter-parts. However, a little is known about the application of the Generalized Gaussian Radial Basis Function on various machine learning algorithms namely, kernel regression, support vector machine (SVM) and pattern-recognition via neural networks. The results that are yielded by Generalized Gaussian RBF in the kernel sense outperforms in stark contrast to Gaussian RBF Kernel, Sigmoid Function and ReLU Function. This manuscript demonstrates the application of the Generalized Gaussian RBF in the kernel sense on the aforementioned machine learning routines along with the comparisons against the aforementioned functions as well.
Evaluation of Dataframe Libraries for Data Preparation on a Single Machine

Data preparation is a trial-and-error process that typically involves countless iterations over the data to define the best pipeline of operators for a given task. With tabular data, practitioners often perform that burdensome activity on local machines by writing ad hoc scripts with libraries based on the Pandas dataframe API and testing them on samples of the entire dataset--the faster the library, the less idle time its users have. In this paper, we evaluate the most popular Python dataframe libraries in general data preparation use cases to assess how they perform on a single machine. To do so, we employ 4 real-world datasets and pipelines with distinct characteristics, covering a variety of scenarios. The insights gained with this experimentation are useful to data scientists who need to choose which of the dataframe libraries best suits their data preparation task at hand. In a nutshell, we found that: for small datasets, Pandas consistently proves to be the best choice with the richest API; when RAM is limited and there is no need to complete compatibility with Pandas API, Polars is the go-to choice thanks to its resource and query optimization; when a GPU is available, CuDF often yields the best performance, while for very large datasets that cannot fit in the GPU memory and RAM, PySpark (thanks to a multi-thread execution and a query optimizer) and Vaex (exploiting a columnar data format) are the best options.
Optimality conditions for the control of a rate independent system with history variable

This paper addresses an optimal control problem governed by a rate independent evolution involving an integral operator. Its particular feature is that the dissipation potential depends on the history of the state. Because of the non-smooth nature of the system, the application of standard adjoint calculus is excluded. We derive optimality conditions in qualified form by approximating the original problem by viscous models. Though these problems preserve the non-smoothness, optimality conditions equivalent to the first-order necessary optimality conditions can be provided in the viscous case. Letting the viscous parameter vanish then yields an optimality system for the original control problem. If the optimal state at the end of the process is not smaller than the desired state, the limit optimality conditions are complete.
Perceptual Musical Features for Interpretable Audio Tagging

In the age of music streaming platforms, the task of automatically tagging music audio has garnered significant attention, driving researchers to devise methods aimed at enhancing performance metrics on standard datasets. Most recent approaches rely on deep neural networks, which, despite their impressive performance, possess opacity, making it challenging to elucidate their output for a given input. While the issue of interpretability has been emphasized in other fields like medicine, it has not received attention in music-related tasks. In this study, we explored the relevance of interpretability in the context of automatic music tagging. We constructed a workflow that incorporates three different information extraction techniques: a) leveraging symbolic knowledge, b) utilizing auxiliary deep neural networks, and c) employing signal processing to extract perceptual features from audio files. These features were subsequently used to train an interpretable machine-learning model for tag prediction. We conducted experiments on two datasets, namely the MTG-Jamendo dataset and the GTZAN dataset. Our method surpassed the performance of baseline models in both tasks and, in certain instances, demonstrated competitiveness with the current state-of-the-art. We conclude that there are use cases where the deterioration in performance is outweighed by the value of interpretability.
MEG Evidence That Modality-Independent Conceptual Representations Encode Visual but Not Lexical Representations

The semantic knowledge stored in our brains can be accessed from different stimulus modalities. For example, a picture of a cat and the word"cat"both engage similar conceptual representations. While existing research has found evidence for modality-independent representations, their content remains unknown. Modality-independent representations could be abstract, or they might be perceptual or even lexical in nature. We used a novel approach combining word/picture cross-condition decoding with neural network classifiers that learned latent modality-independent representations from MEG data. We then compared these representations to models representing semantic, sensory, and lexical features. Results show that modality-independent representations are not strictly amodal; rather, they also contain visual representations. There was no evidence that lexical properties contributed to the representation of modality-independent concepts. These findings support the notion that perceptual processes play a fundamental role in encoding modality-independent conceptual representations. Conversely, lexical representations did not appear to partake in modality-independent semantic knowledge.
Improved Long-Form Speech Recognition By Jointly Modeling The Primary And Non-Primary Speakers

ASR models often suffer from a long-form deletion problem where the model predicts sequential blanks instead of words when transcribing a lengthy audio (in the order of minutes or hours). From the perspective of a user or downstream system consuming the ASR results, this behavior can be perceived as the model “being stuck”, and potentially make the product hard to use. One of the culprits for long-form deletion is training-test data mismatch, which can happen even when the model is trained on diverse and large-scale data collected from multiple application domains. In this work, we introduce a novel technique to simultaneously model different groups of speakers in the audio along with the standard transcript tokens. Speakers are grouped as primary and non-primary, which connects the application domains and significantly alleviates the long-form deletion problem. This improved model neither needs any additional training data nor incurs additional training or inference cost.
VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encoder

This paper introduces a pioneering 3D volumetric encoder designed for text-to-3D generation. To scale up the training data for the diffusion model, a lightweight network is developed to efficiently acquire feature volumes from multi-view images. The 3D volumes are then trained on a diffusion model for text-to-3D generation using a 3D U-Net. This research further addresses the challenges of inaccurate object captions and high-dimensional feature volumes. The proposed model, trained on the public Objaverse dataset, demonstrates promising outcomes in producing diverse and recognizable samples from text prompts. Notably, it empowers finer control over object part characteristics through textual cues, fostering model creativity by seamlessly combining multiple concepts within a single object. This research significantly contributes to the progress of 3D generation by introducing an efficient, flexible, and scalable representation methodology. Code is available at https://github.com/tzco/VolumeDiffusion.