Miracle-dz/UniFAR-FTUs huggingface dataset:
BIM: Block-Wise Self-Supervised Learning with Masked Image Modeling

Like masked language modeling (MLM) in natural language processing, masked image modeling (MIM) aims to extract valuable insights from image patches to enhance the feature extraction capabilities of the underlying deep neural network (DNN). Contrasted with other training paradigms like supervised learning and unsupervised contrastive learning, masked image modeling (MIM) pretraining typically demands significant computational resources in order to manage large training data batches (e.g., 4096). The significant memory and computation requirements pose a considerable challenge to its broad adoption. To mitigate this, we introduce a novel learning framework, termed~\textit{Block-Wise Masked Image Modeling} (BIM). This framework involves decomposing the MIM tasks into several sub-tasks with independent computation patterns, resulting in block-wise back-propagation operations instead of the traditional end-to-end approach. Our proposed BIM maintains superior performance compared to conventional MIM while greatly reducing peak memory consumption. Moreover, BIM naturally enables the concurrent training of numerous DNN backbones of varying depths. This leads to the creation of multiple trained DNN backbones, each tailored to different hardware platforms with distinct computing capabilities. This approach significantly reduces computational costs in comparison with training each DNN backbone individually. Our framework offers a promising solution for resource constrained training of MIM.
Propagate & Distill: Towards Effective Graph Learners Using Propagation-Embracing MLPs

Recent studies attempted to utilize multilayer perceptrons (MLPs) to solve semisupervised node classification on graphs, by training a student MLP by knowledge distillation from a teacher graph neural network (GNN). While previous studies have focused mostly on training the student MLP by matching the output probability distributions between the teacher and student models during distillation, it has not been systematically studied how to inject the structural information in an explicit and interpretable manner. Inspired by GNNs that separate feature transformation $T$ and propagation $\Pi$, we re-frame the distillation process as making the student MLP learn both $T$ and $\Pi$. Although this can be achieved by applying the inverse propagation $\Pi^{-1}$ before distillation from the teacher, it still comes with a high computational cost from large matrix multiplications during training. To solve this problem, we propose Propagate&Distill (P&D), which propagates the output of the teacher before distillation, which can be interpreted as an approximate process of the inverse propagation. We demonstrate that P&D can readily improve the performance of the student MLP.
One-Shot Open Affordance Learning with Foundation Models

We introduce One-shot Open Affordance Learning (OOAL), where a model is trained with just one example per base object category, but is expected to identify novel objects and affordances. While vision-language models excel at recognizing novel objects and scenes, they often struggle to understand finer levels of granularity such as affordances. To handle this issue, we conduct a comprehensive analysis of existing foundation models, to explore their inherent understanding of affordances and assess the potential for data-limited affordance learning. We then propose a vision-language framework with simple and effective designs that boost the alignment between visual features and affordance text embeddings. Experiments on two affordance segmentation benchmarks show that the proposed method outperforms state-of-the-art models with less than 1% of the full training data, and exhibits reasonable generalization capability on unseen objects and affordances.
A Simple Recipe for Language-guided Domain Generalized Segmentation

Generalization to new domains not seen during training is one of the long-standing goals and challenges in deploying neural networks in real-world applications. Existing generalization techniques necessitate substantial data augmentation, potentially sourced from external datasets, and aim at learning invariant representations by imposing various alignment constraints. Large-scale pretraining has recently shown promising generalization capabilities, along with the potential of bridging different modalities. For instance, the recent advent of vision-language models like CLIP has opened the doorway for vision models to exploit the textual modality. In this paper, we introduce a simple framework for generalizing semantic segmentation networks by employing language as the source of randomization. Our recipe comprises three key ingredients: i) the preservation of the intrinsic CLIP robustness through minimal fine-tuning, ii) language-driven local style augmentation, and iii) randomization by locally mixing the source and augmented styles during training. Extensive experiments report state-of-the-art results on various generalization benchmarks. The code will be made available.
On the commutator modulus of continuity for operator monotone functions

Let $f \geq 0$ be operator monotone on $[0, \infty)$. In this paper we prove that for any unitarily-invariant norm $|||-|||$ on $M_n(\mathbb{C})$ and matrices $A, B, X \in M_n(\mathbb{C})$ with $A, B \geq 0$ and $|||X||| \leq 1$, \[|||f(A)X-Xf(B)||| \leq C f(|||AX-XB|||)\] for $C<1.01975$. We do this by reducing this inequality to a function approximation problem and we choose approximate minimizers. This is much progress toward the conjecturally optimal value of $C=1$ which is known only in the case of the Hilbert-Schmidt norm. When $|||-|||$ is the the operator norm $||-||$, we obtain a great reduction of the previously known estimate of $C = 1.25$. We further prove that for $|||X||| \leq 1$, \[|||A^{1/2}X-XB^{1/2}||| \leq 1.00891 |||AX-XB|||. \] This is a great improvement toward the conjecture of G. Pedersen that this inequality for $|||-|||$ being the operator norm holds with $C = 1$. We discuss other related inequalities, including some sharp commutator inequalities. We also prove a sharp equivalence inequality between the operator modulus of continuity and the commutator modulus of continuity for continuous functions on $\mathbb{R}$.
Magnetic field simulations and measurements on the mini-ICAL detector

The ICAL (Iron Calorimeter) is a 51 kton magnetized detector proposed by the INO collaboration. It is designed to detect muons with energies in the 1–20 GeV range. A magnetic field of ∼ 1.5 T in the ICAL detector will be generated by passing a DC current through suitable copper coils. This will enable it to distinguish between μ- and μ+ that will be generated from the interaction of atmospheric νμ and νμ with iron. This will help in resolving the open question of mass ordering in the neutrino sector. Apart from charge identification, the magnetic field will be used to reconstruct the muon momentum (direction and magnitude). Therefore it is important to know the magnetic field in the detector as accurately as possible. We present here an (indirect) measurement of the magnetic field in the 85 ton prototype mini-ICAL detector working in Madurai, Tamil Nadu, for different coil currents. A detailed 3-D finite element simulation was done for the mini-ICAL geometry using Infolytica MagNet software and the magnetic field was computed for different coil currents. This paper presents, for the first time, a comparison of the magnetic field measured in the air gaps with the simulated magnetic field, to validate the simulation using real time data. Using the simulations the magnetic field inside the iron is estimated.
Seifert surfaces in the four-ball and composition of binary quadratic forms

We use composition of binary quadratic forms to systematically create pairs of Seifert surfaces that are non-isotopic in the four-ball. Our main topological result employs Gauss composition to classify the pairs of binary quadratic forms that arise as the Seifert forms of pairs of disjoint Seifert surfaces of genus one. The main ingredient of the proof is number-theoretic and of independent interest. It establishes a new connection between the Bhargava cube and the geometric approach to Gauss composition via planes in the space of two-by-two matrices. In particular, we obtain a geometric recipe that given any two binary quadratic forms finds a Bhargava cube that gives rise to their composition.
Composition schemes: q-enumerations and phase transitions

Composition schemes are ubiquitous in combinatorics, statistical mechanics and probability theory. We give a unifying explanation to various phenomena observed in the combinatorial and statistical physics literature in the context of $q$-enumeration (this is a model where objects with a parameter of value $k$ have a Gibbs measure/Boltzmann weight $q^k$). This leads to phase transition diagrams, highlighting the effect of $q$-enumeration and $q$-distributions on the nature of the limit laws. We build upon the classical distinction between subcritical, critical and supercritical composition schemes, as well as recent results. We apply our results to a wealth of parameters, adding new limit laws to various families of lattice paths and quarter plane walks. We also explain previously observed limit laws for pattern restricted permutations, and a phenomenon observed by Krattenthaler for the wall contacts in watermelons with a wall.
Towards Higher Ranks via Adversarial Weight Pruning

Convolutional Neural Networks (CNNs) are hard to deploy on edge devices due to its high computation and storage complexities. As a common practice for model compression, network pruning consists of two major categories: unstructured and structured pruning, where unstructured pruning constantly performs better. However, unstructured pruning presents a structured pattern at high pruning rates, which limits its performance. To this end, we propose a Rank-based PruninG (RPG) method to maintain the ranks of sparse weights in an adversarial manner. In each step, we minimize the low-rank approximation error for the weight matrices using singular value decomposition, and maximize their distance by pushing the weight matrices away from its low rank approximation. This rank-based optimization objective guides sparse weights towards a high-rank topology. The proposed method is conducted in a gradual pruning fashion to stabilize the change of rank during training. Experimental results on various datasets and different tasks demonstrate the effectiveness of our algorithm in high sparsity. The proposed RPG outperforms the state-of-the-art performance by 1.13% top-1 accuracy on ImageNet in ResNet-50 with 98% sparsity. The codes are available at https://github.com/huawei-noah/Efficient-Computing/tree/master/Pruning/RPG and https://gitee.com/mindspore/models/tree/master/research/cv/RPG.
A Morse theoretical approach to Fourier transforms of holonomic D-modules in dimension one

We study Fourier transforms of holonomic D-modules on the affine line and show that their enhanced solution complexes are described by a twisted Morse theory. We thus recover and even strengthen the well-known formula for their exponential factors i.e. the stationary phase method. As a byproduct, natural bases of their holomorphic solutions are also obtained via rapid decay cycles.
Opposite Effects of the Rotational and Translational Energy on the Rates of Ion-Molecule Reactions near 0 K: The 
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:msubsup><mml:mrow><mml:mi mathvariant="normal">D</mml:mi></mml:mrow><mml:mn>2</mml:mn><mml:mrow><

The ion-molecule reactions $\text{D}_2^++\text{NH}_3$ and $\text{D}_2^++\text{ND}_3$ are studied at low collision energies ($E_{\text{coll}}$ from zero to $\sim k_\textrm{B}\cdot 50\,\text{K}$), with the $\text{D}_2^+$ ions in the ground rovibrational state and for different rotational temperatures of the ammonia molecules, using the Rydberg-Stark merged-beam approach. Two different rotational temperatures ($\sim\,15\,\text{K}$ and $\sim\,40\,\text{K}$), measured by (2+1) resonance-enhanced multiphoton-ionization spectroscopy, are obtained by using a seeded supersonic expansion in He and a pure ammonia expansion, respectively. The experimental data reveal a strong enhancement of the rate coefficients at the lowest collision energies caused by the charge-dipole interaction. Calculations based on a rotationally adiabatic capture model accurately reproduce the observed kinetic-energy dependence of the rate coefficients. The rate coefficients increase with increasing rotational temperature of the ammonia molecules, which contradicts the expectation that rotational excitation should average the dipoles out. Moreover, these reactions exhibit a pronounced inverse kinetic isotope effect. The difference is caused by nuclear-spin-statistical factors, and the smaller rotational constants and tunneling splittings in $\text{ND}_3$.
Continual Learning for Image Segmentation with Dynamic Query

Image segmentation based on continual learning exhibits a critical drop of performance, mainly due to catastrophic forgetting and background shift, as they are required to incorporate new classes continually. In this paper, we propose a simple, yet effective Continual Image Segmentation method with incremental Dynamic Query (CISDQ), which decouples the representation learning of both old and new knowledge with lightweight query embedding. CISDQ mainly includes three contributions: 1) We define dynamic queries with adaptive background class to exploit past knowledge and learn future classes naturally. 2) CISDQ proposes a class/instance-aware Query Guided Knowledge Distillation strategy to overcome catastrophic forgetting by capturing the inter-class diversity and intra-class identity. 3) Apart from semantic segmentation, CISDQ introduce the continual learning for instance segmentation in which instance-wise labeling and supervision are considered. Extensive experiments on three datasets for two tasks (i.e., continual semantic and instance segmentation are conducted to demonstrate that CISDQ achieves the state-of-the-art performance, specifically, obtaining 4.4% and 2.9% mIoU improvements for the ADE 100-10 (6 steps) setting and ADE 100-5 (11 steps) setting.
SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action Segmentation

Multi-modal human action segmentation is a critical and challenging task with a wide range of applications. Nowadays, the majority of approaches concentrate on the fusion of dense signals (i.e., RGB, optical flow, and depth maps). However, the potential contributions of sparse IoT sensor signals, which can be crucial for achieving accurate recognition, have not been fully explored. To make up for this, we introduce a Sparse signalguided Transformer (SigFormer) to combine both dense and sparse signals. We employ mask attention to fuse localized features by constraining cross-attention within the regions where sparse signals are valid. However, since sparse signals are discrete, they lack sufficient information about the temporal action boundaries. Therefore, in SigFormer, we propose to emphasize the boundary information at two stages to alleviate this problem. In the first feature extraction stage, we introduce an intermediate bottleneck module to jointly learn both category and boundary features of each dense modality through the inner loss functions. After the fusion of dense modalities and sparse signals, we then devise a two-branch architecture that explicitly models the interrelationship between action category and temporal boundary. Experimental results demonstrate that SigFormer outperforms the state-of-the-art approaches on a multi-modal action segmentation dataset from real industrial environments, reaching an outstanding F1 score of 0.958. The codes and pre-trained models have been available at https://github.com/LIUQI-creat/SigFormer.
Finiteness of hyperbolic entropy for holomorphic foliations with non-degenerate singularities

Consider $\mathscr{F}=(M,\mathscr{L},E)$ a Brody-hyperbolic foliation on a compact complex surface $M$. Suppose that the singularities of $\mathscr{F}$ are all non-degenerate. We show that the hyperbolic entropy of $\mathscr{F}$ is finite.
Analyzing and Explaining Image Classifiers via Diffusion Guidance

While deep learning has led to huge progress in complex image classification tasks like ImageNet, unexpected failure modes, e.g. via spurious features, call into question how reliably these classifiers work in the wild. Furthermore, for safety-critical tasks the black-box nature of their decisions is problematic, and explanations or at least methods which make decisions plausible are needed urgently. In this paper, we address these problems by generating images that optimize a classifier-derived objective using a framework for guided image generation. We analyze the behavior and decisions of image classifiers by visual counterfactual explanations (VCEs), detection of systematic mistakes by analyzing images where classifiers maximally disagree, and visualization of neurons to verify potential spurious features. In this way, we validate existing observations, e.g. the shape bias of adversarially robust models, as well as novel failure modes, e.g. systematic errors of zero-shot CLIP classifiers, or identify harmful spurious features. Moreover, our VCEs outperform previous work while being more versatile.
A Categorical Framework for Quantifying Emergent Effects in Network Topology

Emergent effect is crucial to the understanding of the properties of complex systems that do not appear in their basic units, but there has been a lack of theories to measure and understand its mechanisms. In this paper, we established a framework based on homological algebra that encodes emergence as the mathematical structure of cohomologies and then applied it to network models to develop a computational measure of emergence. This framework ties the emergence of a system to its network topology and local structures, paving the way to predict and understand the cause of emergent effects. We show in our numerical experiment that our measure of emergence correlates with the existing information-theoretic measure of emergence.
Are ensembles getting better all the time?

Ensemble methods combine the predictions of several base models. We study whether or not including more models always improves their average performance. This question depends on the kind of ensemble considered, as well as the predictive metric chosen. We focus on situations where all members of the ensemble are a priori expected to perform as well, which is the case of several popular methods such as random forests or deep ensembles. In this setting, we show that ensembles are getting better all the time if, and only if, the considered loss function is convex. More precisely, in that case, the average loss of the ensemble is a decreasing function of the number of models. When the loss function is nonconvex, we show a series of results that can be summarised as: ensembles of good models keep getting better, and ensembles of bad models keep getting worse. To this end, we prove a new result on the monotonicity of tail probabilities that may be of independent interest. We illustrate our results on a medical prediction problem (diagnosing melanomas using neural nets) and a"wisdom of crowds"experiment (guessing the ratings of upcoming movies).
Variational Bayes image restoration with compressive autoencoders

Regularization of inverse problems is of paramount importance in computational imaging. The ability of neural networks to learn efficient image representations has been recently exploited to design powerful data-driven regularizers. While state-of-the-art plug-and-play methods rely on an implicit regularization provided by neural denoisers, alternative Bayesian approaches consider Maximum A Posteriori (MAP) estimation in the latent space of a generative model, thus with an explicit regularization. However, state-of-the-art deep generative models require a huge amount of training data compared to denoisers. Besides, their complexity hampers the optimization involved in latent MAP derivation. In this work, we first propose to use compressive autoencoders instead. These networks, which can be seen as variational autoencoders with a flexible latent prior, are smaller and easier to train than state-of-the-art generative models. As a second contribution, we introduce the Variational Bayes Latent Estimation (VBLE) algorithm, which performs latent estimation within the framework of variational inference. Thanks to a simple yet efficient parameterization of the variational posterior, VBLE allows for fast and easy (approximate) posterior sampling. Experimental results on image datasets BSD and FFHQ demonstrate that VBLE reaches similar performance than state-of-the-art plug-and-play methods, while being able to quantify uncertainties faster than other existing posterior sampling techniques.
Non-Visible Light Data Synthesis and Application: A Case Study for Synthetic Aperture Radar Imagery

We explore the"hidden"ability of large-scale pre-trained image generation models, such as Stable Diffusion and Imagen, in non-visible light domains, taking Synthetic Aperture Radar (SAR) data for a case study. Due to the inherent challenges in capturing satellite data, acquiring ample SAR training samples is infeasible. For instance, for a particular category of ship in the open sea, we can collect only few-shot SAR images which are too limited to derive effective ship recognition models. If large-scale models pre-trained with regular images can be adapted to generating novel SAR images, the problem is solved. In preliminary study, we found that fine-tuning these models with few-shot SAR images is not working, as the models can not capture the two primary differences between SAR and regular images: structure and modality. To address this, we propose a 2-stage low-rank adaptation method, and we call it 2LoRA. In the first stage, the model is adapted using aerial-view regular image data (whose structure matches SAR), followed by the second stage where the base model from the first stage is further adapted using SAR modality data. Particularly in the second stage, we introduce a novel prototype LoRA (pLoRA), as an improved version of 2LoRA, to resolve the class imbalance problem in SAR datasets. For evaluation, we employ the resulting generation model to synthesize additional SAR data. This augmentation, when integrated into the training process of SAR classification as well as segmentation models, yields notably improved performance for minor classes