Miracle-dz/UniFAR-FTUs huggingface dataset:
Crowd Counting with Online Knowledge Learning

Efficient crowd counting models are urgently required for the applications in scenarios with limited computing resources, such as edge computing and mobile devices. A straightforward method to achieve this is knowledge distillation (KD), which involves using a trained teacher network to guide the training of a student network. However, this traditional two-phase training method can be time-consuming, particularly for large datasets, and it is also challenging for the student network to mimic the learning process of the teacher network. To overcome these challenges, we propose an online knowledge learning method for crowd counting. Our method builds an end-to-end training framework that integrates two independent networks into a single architecture, which consists of a shared shallow module, a teacher branch, and a student branch. This approach is more efficient than the two-stage training technique of traditional KD. Moreover, we propose a feature relation distillation method which allows the student branch to more effectively comprehend the evolution of inter-layer features by constructing a new inter-layer relationship matrix. It is combined with response distillation and feature internal distillation to enhance the transfer of mutually complementary information from the teacher branch to the student branch. Extensive experiments on four challenging crowd counting datasets demonstrate the effectiveness of our method which achieves comparable performance to state-of-the-art methods despite using far fewer parameters.
High-Frequency Volatility Estimation with Fast Multiple Change Points Detection

We propose high-frequency volatility estimators with multiple change points that are $\ell_1$-regularized versions of two classical estimators: quadratic variation and bipower variation. We establish consistency of these estimators for the true unobserved volatility and the change points locations under general sub-Weibull distribution assumptions on the jump process. The proposed estimators employ the computationally efficient least angle regression algorithm for estimation purposes, followed by a reduced dynamic programming step to refine the final number of change points. In terms of numerical performance, the proposed estimators are computationally fast and accurately identify breakpoints near the end of the sample, which is highly desirable in today's electronic trading environment. In terms of out-of-sample volatility prediction, our new estimators provide more realistic and smoother volatility forecasts, and they outperform a wide range of classical and recent volatility estimators across various frequencies and forecasting horizons.
Bounded twin-width graphs are polynomially χ-bounded

We show that every graph with twin-width $t$ has chromatic number $O(\omega ^{k_t})$ for some integer $k_t$, where $\omega$ denotes the clique number. This extends a quasi-polynomial bound from Pilipczuk and Soko{\l}owski and generalizes a result for bounded clique-width graphs by Bonamy and Pilipczuk. The proof uses the main ideas of the quasi-polynomial approach, with a different treatment of the decomposition tree. In particular, we identify two types of extensions of a class of graphs: the delayed-extension (which preserves polynomial $\chi$-boundedness) and the right-extension (which preserves polynomial $\chi$-boundedness under bounded twin-width condition). Our main result is that every bounded twin-width graph is a delayed extension of simpler classes of graphs, each expressed as a bounded union of right extensions of lower twin-width graphs.
Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization

This work studies the generalization error of gradient methods. More specifically, we focus on how training steps $T$ and step-size $\eta$ might affect generalization in smooth stochastic convex optimization (SCO) problems. We first provide tight excess risk lower bounds for Gradient Descent (GD) and Stochastic Gradient Descent (SGD) under the general non-realizable smooth SCO setting, suggesting that existing stability analyses are tight in step-size and iteration dependence, and that overfitting provably happens. Next, we study the case when the loss is realizable, i.e. an optimal solution minimizes all the data points. Recent works show better rates can be attained but the improvement is reduced when training time is long. Our paper examines this observation by providing excess risk lower bounds for GD and SGD in two realizable settings: 1) $\eta T = \bigO{n}$, and (2) $\eta T = \bigOmega{n}$, where $n$ is the size of dataset. In the first case $\eta T = \bigOmega{n}$, our lower bounds tightly match and certify the respective upper bounds. However, for the case $\eta T = \bigOmega{n}$, our analysis indicates a gap between the lower and upper bounds. A conjecture is proposed that the gap can be closed by improving upper bounds, supported by analyses in two special scenarios.
Decomposed Prototype Learning for Few-Shot Scene Graph Generation

Today's scene graph generation (SGG) models typically require abundant manual annotations to learn new predicate types. Thus, it is difficult to apply them to real-world applications with a long-tailed distribution of predicates. In this paper, we focus on a new promising task of SGG: few-shot SGG (FSSGG). FSSGG encourages models to be able to quickly transfer previous knowledge and recognize novel predicates well with only a few examples. Although many advanced approaches have achieved great success on few-shot learning (FSL) tasks, straightforwardly extending them into FSSGG is not applicable due to two intrinsic characteristics of predicate concepts: 1) Each predicate category commonly has multiple semantic meanings under different contexts. 2) The visual appearance of relation triplets with the same predicate differs greatly under different subject-object pairs. Both issues make it hard to model conventional latent representations for predicate categories with state-of-the-art FSL methods. To this end, we propose a novel Decomposed Prototype Learning (DPL). Specifically, we first construct a decomposable prototype space to capture intrinsic visual patterns of subjects and objects for predicates, and enhance their feature representations with these decomposed prototypes. Then, we devise an intelligent metric learner to assign adaptive weights to each support sample by considering the relevance of their subject-object pairs. We further re-split the VG dataset and compare DPL with various FSL methods to benchmark this task. Extensive results show that DPL achieves excellent performance in both base and novel categories.
Textless Speech-to-Music Retrieval Using Emotion Similarity

We introduce a framework that recommends music based on the emotions of speech. In content creation and daily life, speech contains information about human emotions, which can be enhanced by music. Our framework focuses on a cross-domain retrieval system to bridge the gap between speech and music via emotion labels. We explore different speech representations and report their impact on different speech types, including acting voice and wake-up words. We also propose an emotion similarity regularization term in cross-domain retrieval tasks. By incorporating the regularization term into training, similar speech-and-music pairs in the emotion space are closer in the joint embedding space. Our comprehensive experimental results show that the proposed model is effective in textless speech-to-music retrieval.
Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process

We develop a generalized 3D shape generation prior model, tailored for multiple 3D tasks including unconditional shape generation, point cloud completion, and cross-modality shape generation, etc. On one hand, to precisely capture local fine detailed shape information, a vector quantized variational autoencoder (VQ-VAE) is utilized to index local geometry from a compactly learned code-book based on a broad set of task training data. On the other hand, a discrete diffusion generator is introduced to model the inherent structural dependencies among different tokens. In the meantime, a multi-frequency fusion module (MFM) is developed to suppress high-frequency shape feature fluctuations, guided by multi-frequency contextual information. The above designs jointly equip our proposed 3D shape prior model with high-fidelity, diverse features as well as the capability of cross-modality alignment, and extensive experiments have demonstrated superior performances on various 3D shape generation tasks.
Knowledge Distillation from Multiple Foundation Models for End-to-End Speech Recognition

Although large foundation models pre-trained by self-supervised learning have achieved state-of-the-art performance in many tasks including automatic speech recognition (ASR), knowledge distillation (KD) is often required in practice to transfer the knowledge learned by large teacher models into much smaller student models with affordable computation and memory costs. This paper proposes a novel two-stage KD framework to distil the knowledge from multiple speech foundation models as teachers into a single student neural transducer model for ASR. In the first stage, the student model encoder is pre-trained using the embeddings extracted from multiple teacher models. In the second stage, the student encoder is fine-tuned with the audio-text pairs based on the ASR task. Experiments on the LibriSpeech 100-hour subset show that the proposed KD framework improves the performance of both streaming and non-streaming student models when using only one teacher. The performance of the student model can be further enhanced when multiple teachers are used jointly, achieving word error rate reductions (WERRs) of 17.5% and 10.6%. Our proposed framework can be combined with other existing KD methods to achieve further improvements. Further WERRs were obtained by incorporating extra unlabelled data during encoder pre-training, leading to a total relative WERR of 55.0% on the non-streaming student model.
A Multi-Level Fast-Marching Method For The Minimum Time Problem

We introduce a new numerical method to approximate the solutions of a class of stationary Hamilton-Jacobi (HJ) partial differential equations arising from minimum time optimal control problems. We rely on several grid approximations, and look for the optimal trajectories by using the coarse grid approximations to reduce the search space in fine grids. This may be thought of as an infinitesimal version of the ''highway hierarchy'' method which has been developed to solve shortest path problems (with discrete time and discrete state). We obtain, for each level, an approximate value function on a sub-domain of the state space. We show that the sequence obtained in this way does converge to the viscosity solution of the HJ equation. Moreover, for our multi-level algorithm, if $0<\gamma \leq 1$ is the convergence rate of the classical numerical scheme, then the number of arithmetic operations needed to obtain an error in $O(\varepsilon)$ is in $\widetilde{O}(\varepsilon^{-\frac{\theta d}{\gamma}})$, to be compared with $\widetilde{O}(\varepsilon^{-\frac{d}{\gamma}})$ for ordinary grid-based methods. Here $d$ is the dimension of the problem, and $\theta<1$ depends on $d$ and on the ''stiffness"of the value function around optimal trajectories, and the notation $\widetilde{O}$ ignores logarithmic factors. When $\gamma =1$ and the stiffness is high, this complexity becomes in $\widetilde{O}(\varepsilon^{-1})$. We illustrate the approach by solving HJ equations of eikonal type up to dimension 7.
A p-adic Cartier isomorphism between the A_inf-cohomology and de Rham-Witt complexes for semistable formal schemes

\v{C}esnavi\v{c}ius-Koshikawa constructed the A_inf-cohomology theory for semistable formal schemes over the ring of integers of C_p. We prove the p-adic Cartier isomorphism between the A_inf-cohomology and de Rham-Witt complexes for semistable formal schemes, extending the result of Bhatt-Morrow-Scholze in the smooth case.
A Content Adaptive Learnable "Time-Frequency" Representation for audio Signal Processing

We propose a learnable content adaptive front end for audio signal processing. Before the modern advent of deep learning, we used fixed representation non-learnable front-ends like spectrogram or mel-spectrogram with/without neural architectures. With convolutional architectures supporting various applications such as ASR and acoustic scene understanding, a shift to a learnable front ends occurred in which both the type of basis functions and the weight were learned from scratch and optimized for the particular task of interest. With the shift to transformer-based architectures with no convolutional blocks present, a linear layer projects small waveform patches onto a small latent dimension before feeding them to a transformer architecture. In this work, we propose a way of computing a content-adaptive learnable time-frequency representation. We pass each audio signal through a bank of convolutional filters, each giving a fixed-dimensional vector. It is akin to learning a bank of finite impulse-response filterbanks and passing the input signal through the optimum filter bank depending on the content of the input signal. A content-adaptive learnable time-frequency representation may be more broadly applicable, beyond the experiments in this paper.
Pairing of Composite-Electrons and Composite-Holes in $\nu_T=1$ Quantum Hall Bilayers

Motivated by recent experimental indications of preformed electron-hole pairs in $\nu_T=1$ quantum Hall bilayers at relatively large separation, we formulate a Chern-Simons (CS) theory of the coupled composite electron liquid (CEL) and composite hole liquid (CHL). We show that the effective action of the CS gauge field fluctuations around the saddle-point leads to stable pairing between CEL and CHL. We find that the CEL-CHL pairing theory leads to a dominant $s$-wave channel in contrast to the dominant $p$-wave channel found in the CEL-CEL pairing theory. Moreover, the CEL-CHL pairing is generally stronger than the CEL-CEL pairing across the whole frequency spectrum. Finally, we discuss possible differences between the two pairing mechanisms that may be probed in experiments.
Extraordinary surface critical behavior induced by the symmetry-protected topological states of a two-dimensional quantum magnet

Using Quantum Monte Carlo simulations, we study spin-1/2 diagonal ladders coupled by ferromagnetic Heisenberg interactions. The model can also be viewed as usual ladders with ferromagnetic rung couplings coupled by antiferromagnetic diagonal couplings. We find that the model hosts a striped magnetic ordered phase and two topological nontrivial Haldane phases, separated by two quantum critical points. We show that the two quantum critical points are all in the three-dimensional O(3) universality class irrelevant to the topological properties of the Haldane phases. The properties of the surface formed by ladder ends in the two Haldane phases are studied. We find that the surface states are both gapless due to the symmetry-protected topological bulk states. We further demonstrate that extraordinary surface critical behaviors are realized at both critical points on such gapless surfaces without enhancing the surface coupling. Notably, the surface is not expected to be ordered in the three-dimensional classical O(3) critical point, suggesting that the topological properties of the Haldane phases are responsible for such surface critical behavior.
Multiscale Audio Spectrogram Transformer for Efficient Audio Classification

Audio event has a hierarchical architecture in both time and frequency and can be grouped together to construct more abstract semantic audio classes. In this work, we develop a multiscale audio spectrogram Transformer (MAST) that employs hierarchical representation learning for efficient audio classification. Specifically, MAST employs one-dimensional (and two-dimensional) pooling operators along the time (and frequency domains) in different stages, and progressively reduces the number of tokens and increases the feature dimensions. MAST significantly outperforms AST [1] by 22.2%, 4.4% and 4.7% on Kinetics-Sounds, Epic-Kitchens-100 and VGGSound in terms of the top-1 accuracy without external training data. On the downloaded AudioSet dataset, which has over 20% missing audios, MAST also achieves slightly better accuracy than AST. In addition, MAST is 5× more efficient in terms of multiply-accumulates (MACs) with 42% reduction in the number of parameters compared to AST. Through clustering metrics and visualizations, we demonstrate that the proposed MAST can learn semantically more separable feature representations from audio signals.
Positional Diffusion: Ordering Unordered Sets with Diffusion Probabilistic Models

Positional reasoning is the process of ordering unsorted parts contained in a set into a consistent structure. We present Positional Diffusion, a plug-and-play graph formulation with Diffusion Probabilistic Models to address positional reasoning. We use the forward process to map elements' positions in a set to random positions in a continuous space. Positional Diffusion learns to reverse the noising process and recover the original positions through an Attention-based Graph Neural Network. We conduct extensive experiments with benchmark datasets including two puzzle datasets, three sentence ordering datasets, and one visual storytelling dataset, demonstrating that our method outperforms long-lasting research on puzzle solving with up to +18% compared to the second-best deep learning method, and performs on par against the state-of-the-art methods on sentence ordering and visual storytelling. Our work highlights the suitability of diffusion models for ordering problems and proposes a novel formulation and method for solving various ordering tasks. Project website at https://iit-pavis.github. io/Positional_Diffusion/
Aerothermodynamic Simulators for Rocket Design using Neural Fields

The typical size of computational meshes needed for realistic geometries and high-speed flow conditions makes Computational Fluid Dynamics (CFD) impractical for full-mission performance prediction and control. Reduced-Order Models (ROMs) in low-speed aerodynamics have come a long way in terms of reconstructing coherent flow patterns, thus enabling aerodynamic performance prediction. While many examples of ROMs exist for low-speed aerodynamics, there is no such broad literature for high-speed flows. We propose to use physics-enhanced neural fields for prediction of the steady, supersonic flow around a rocket for resolving: the bow shock profile, the boundary layer gradients and the wake over a range of incidences. This approach can lead to the construction of a computationally inexpensive, continuous aerothermodynamic model of a rocket at realistic flight conditions with applications to aerodynamic design and mission analysis and control. The use of neural fields allows to quickly and efficiently sweep the Angle of Attack (AoA) in a continuous manner, as compared to traditional CFD which requires running simulations for each discrete incidence value.
Audio-Text Models Do Not Yet Leverage Natural Language

Multi-modal contrastive learning techniques in the audio-text domain have quickly become a highly active area of research. Most works are evaluated with standard audio retrieval and classification benchmarks assuming that (i) these models are capable of leveraging the rich information contained in natural language, and (ii) current benchmarks are able to capture the nuances of such information. In this work, we show that state-of-the-art audio-text models do not yet really understand natural language, especially contextual concepts such as sequential or concurrent ordering of sound events. Our results suggest that existing benchmarks are not sufficient to assess these models' capabilities to match complex contexts from the audio and text modalities. We propose a Transformer-based architecture and show that, unlike prior work, it is capable of modeling the sequential relationship between sound events in the text and audio, given appropriate benchmark data. We advocate for the collection or generation of additional, diverse, data to allow future research to fully leverage natural language for audio-text modeling.
A quasi‐dynamic procedure for coupled thermal simulations

This paper outlines the development and adaptation of a coupling strategy for transient temperature analysis in a solid via a conjugate heat transfer method. This study proposes a quasi‐dynamic coupling procedure to bridge the temporal disparities between the fluid and the solid. In this approach, dynamic thermal modeling in the solid is coupled with a sequence of steady states in the fluid. This quasi‐dynamic algorithm has been applied to the problem of convective heat transfer over, and transient conduction heat transfer within, a flat plate using the severe thermal conditions of a solid propellant rocket. Two different coupled thermal computations have been performed. In the first one—referred to as the reference computation—the coupling period is equal to the smallest solid time constant. In the second one, a very large coupling period is used. The results show that the procedure can predict accurate transient temperature fields at a reasonable computational cost. The simulation CPU time is approximately reduced by up to 90%, while maintaining a very good accuracy. All the details of the numerical test case are given in the paper. This application illustrates the capabilities and the overall efficiency of this coupled approach in a solid transient problem using long term simulations of time dependent flows. Copyright © 2013 John Wiley & Sons, Ltd.