Miracle-dz/UniFAR-FTUs huggingface dataset:
Token Pooling in Vision Transformers

Despite the recent success in many applications, the high computational requirements of vision transformers limit their use in resource-constrained settings. While many existing methods improve the quadratic complexity of attention, in most vision transformers, self-attention is not the major computation bottleneck, e.g., more than 80% of the computation is spent on fully-connected layers. To improve the computational complexity of all layers, we propose a novel token downsampling method, called Token Pooling, efficiently exploiting redundancies in the images and intermediate token representations. We show that, under mild assumptions, softmax-attention acts as a high-dimensional low-pass (smoothing) filter. Thus, its output contains redundancy that can be pruned to achieve a better trade-off between the computational cost and accuracy. Our new technique accurately approximates a set of tokens by minimizing the reconstruction error caused by downsampling. We solve this optimization problem via cost-efficient clustering. We rigorously analyze and compare to prior downsampling methods. Our experiments show that Token Pooling significantly improves the cost-accuracy trade-off over the state-of-the-art downsampling. Token Pooling is a simple and effective operator that can benefit many architectures. Applied to DeiT, it achieves the same ImageNet top-1 accuracy using 42% fewer computations.
Towards explainable real estate valuation via evolutionary algorithms

Human lives are increasingly influenced by algorithms, which therefore need to meet higher standards not only in accuracy but also with respect to explainability. This is especially true for high-stakes areas such as real estate valuation. Unfortunately, the methods applied there often exhibit a trade-off between accuracy and explainability. One explainable approach is case-based reasoning (CBR), where each decision is supported by specific previous cases. However, such methods can be wanting in accuracy. The unexplainable machine learning approaches are often observed to provide higher accuracy but are not scrutable in their decision-making. In this paper, we apply evolutionary algorithms (EAs) to CBR predictors in order to improve their performance. In particular, we deploy EAs to the similarity functions (used in CBR to find comparable cases), which are fitted to the data set at hand. As a consequence, we achieve higher accuracy than state-of-the-art deep neural networks (DNNs), while keeping interpretability and explainability. These results stem from our empirical evaluation on a large data set of real estate offers where we compare known similarity functions, their EA-improved counterparts, and DNNs. Surprisingly, DNNs are only on par with standard CBR techniques. However, using EA-learned similarity functions does yield an improved performance.
6D-ViT: Category-Level 6D Object Pose Estimation via Transformer-Based Instance Representation Learning

This paper presents 6D vision transformer (6D-ViT), a transformer-based instance representation learning network suitable for highly accurate category-level object pose estimation based on RGB-D images. Specifically, a novel two-stream encoder-decoder framework is dedicated to exploring complex and powerful instance representations from RGB images, point clouds, and categorical shape priors. The whole framework consists of two main branches, named Pixelformer and Pointformer. Pixelformer contains a pyramid transformer encoder with an all-multilayer perceptron (MLP) decoder to extract pixelwise appearance representations from RGB images, while Pointformer relies on a cascaded transformer encoder and an all-MLP decoder to acquire the pointwise geometric characteristics from point clouds. Then, dense instance representations (i.e., correspondence matrix and deformation field) for NOCS model reconstruction are obtained from a multisource aggregation (MSA) network with shape prior, appearance and geometric information as inputs. Finally, the instance 6D pose is computed by solving the similarity transformation between the observed point clouds and the reconstructed NOCS representations. Extensive experiments with synthetic and real-world datasets demonstrate that the proposed framework achieves state-of-the-art performance for both datasets. Code is available at https://github.com/luzzou/6D-ViT.
An efficient heterogeneous gold(I)-catalyzed intermolecular cycloaddition of 2-aminoaryl carbonyls and internal alkynes leading to polyfunctionalized quinolines

Abstract The first heterogeneous intermolecular cycloaddition of 2-aminoaryl carbonyls and internal alkynes was realized in DMF at 100 °C by using a triphenylphosphine-functionalized MCM-41-supported gold(I) complex [MCM-41-PPh3-AuCl] and AgOTf as catalysts, yielding a variety of polyfunctionalized quinolines in good to excellent yields. This heterogeneous gold(I) complex could easily be prepared via a simple two-step procedure from commercially available reagents and recovered by filtration of the reaction mixture. The recovered catalyst could be reused at least seven times with almost consistent activity without addition of AgOTf as a cocatalyst. Graphical Abstract
J-PLUS: Detecting and studying extragalactic globular clusters - The case of NGC 1023

Extragalactic globular clusters (GCs) are key objects for studying the history of galaxies. The arrival of wide-field surveys such as the Javalambre Photometric Local Universe Survey (J-PLUS) offers new possibilities for the study of these systems. We perform the first study of GCs in J-PLUS to recover information about the history of NGC 1023 taking advantage of wide-field images and 12 filters. We develop the semiautomatic pipeline GCFinder that detects GC candidates in J-PLUS images and can also be adapted to similar surveys. We study the stellar population properties of a sub-sample of GC candidates using spectral energy distribution (SED) fitting. We find 523 GC candidates in NGC 1023, of which $\sim$300 are new. We identify subpopulations of GC candidates, where age and metallicity distributions have multiple peaks. By comparing our results with simulations, we report a possible broad age-metallicity relation, evidence that NGC 1023 experienced accretion events in the past. The dominating age peak is at $10^{10}$ yr. We report a correlation between masses and ages that suggests that massive GC candidates are more likely to survive the turbulent history of the host galaxy. Modeling the light of NGC 1023, we find two spiral-like arms and detect a displacement of the galaxy's photometric center with respect to the outer isophotes and center of GC distribution ($\sim$700 pc and $\sim$1600 pc, respectively), which could be the result of ongoing interaction between NGC 1023 and NGC 1023A. By studying the GC system of NGC 1023 with J-PLUS we showcase the power of multi-band surveys for this kind of study and find evidence of a complex accretion history of the host galaxy.
Competitive Perimeter Defense of Conical Environments

We consider a perimeter defense problem in a planar conical environment in which a single vehicle, having a finite capture radius, aims to defend a concentric perimeter from mobile intruders. The intruders are arbitrarily released at the circumference of the environment and they move radially toward the perimeter with fixed speed. We present a competitive analysis approach to this problem by measuring the performance of multiple online algorithms for the vehicle against arbitrary inputs, relative to an optimal offline algorithm that has information about entire input instance in advance. In particular, we establish two necessary conditions on the parameter space to guarantee (i) finite competitiveness of any algorithm and (ii) a competitive ratio of at least 2 for any algorithm. We then design and analyze three online algorithms and characterize parameter regimes in which they have finite competitive ratios. Specifically, our first two algorithms are provably 1, and 2-competitive, respectively, whereas our third algorithm exhibits different competitive ratios in different regimes of problem parameters. Finally, we provide a numerical plot in the parameter space to reveal additional insights into the relative performance of our algorithms.
A derived construction of eigenvarieties

We construct a derived variant of Emerton’s eigenvarieties using the locally analytic representation theory of p -adic groups. The main innova- tions include comparison and exploitation of two homotopy equivalent completed complexes associated to the locally symmetric spaces of a quasi-split reductive group G , comparison to overconvergent cohomology, proving exactness of ﬁnite slope part functor, together with some representation-theoretic statements. As a global application, we exhibit an eigenvariety coming from data of GL n over a CM ﬁeld as a subeigenvariety for a quasi-split unitary group.
Straw mulching with fertilizer nitrogen: An approach for improving crop yield, soil nutrients and enzyme activities

Field experiments were conducted to study soil properties, soil enzymes activities, water use efficiency (WUE) and crop productivity after six years of soya bean straw mulching in the semi‐arid conditions of China. The experiment included four treatments: CK (Control), N (240 kg N ha‐1), H (soya bean straw mulching at half rate 700 kg ha‐1 with 240 kg N ha‐1) and F (soya bean straw mulching at full rate 1,400 kg ha‐1 with 240 kg N ha‐1). Soil organic carbon (SOC), soil labile organic carbon (LOC), soil available N (AN), available P (AP) and enzyme activities were analysed after wheat harvesting in 2016 and 2017. Results show that straw amounts had positive effects on the soil fertility indices being higher for treatment F. The SOC, LOC, AN, AP and enzyme activities (i.e. saccharase, urease and alkaline phosphatase) were in the order of F > H > N > CK. High wheat grain yield and WUE were observed for F treatment. A total of six years mulching along with 240 kg ha‐1 nitrogen fertilizer application is sufficient for wheat yield stability and improving soil properties except urease activities in the semi‐arid condition of China. However, the straw mulching amount should be further studied with minimum nitrogen fertilizer for an environment‐friendly and effective approach for improving the soil biological properties with adequate crop production on a sustainable basis in the semi‐arid region of China.
The One Step Malliavin scheme: new discretization of BSDEs implemented with deep learning regressions

A novel discretization is presented for decoupled forward–backward stochastic differential equations (FBSDE) with differentiable coefficients, simultaneously solving the BSDE and its Malliavin sensitivity problem. The control process is estimated by the corresponding linear BSDE driving the trajectories of the Malliavin derivatives of the solution pair, which implies the need to provide accurate $\varGamma $ estimates. The approximation is based on a merged formulation given by the Feynman–Kac formulae and the Malliavin chain rule. The continuous time dynamics is discretized with a theta-scheme. In order to allow for an efficient numerical solution of the arising semidiscrete conditional expectations in possibly high dimensions, it is fundamental that the chosen approach admits to differentiable estimates. Two fully-implementable schemes are considered: the BCOS method as a reference in the one-dimensional framework and neural network Monte Carlo regressions in case of high-dimensional problems, similarly to the recently emerging class of Deep BSDE methods (Han et al. (2018 Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci., 115, 8505–8510); Huré et al. (2020 Deep backward schemes for high-dimensional nonlinear PDEs. Math. Comp., 89, 1547–1579)). An error analysis is carried out to show $\mathbb{L}^2$ convergence of order $1/2$, under standard Lipschitz assumptions and additive noise in the forward diffusion. Numerical experiments are provided for a range of different semilinear equations up to $50$ dimensions, demonstrating that the proposed scheme yields a significant improvement in the control estimations.
Consistency of the full and reduced order models for evolve‐filter‐relax regularization of convection‐dominated, marginally‐resolved flows

Numerical stabilization is often used to eliminate (alleviate) the spurious oscillations generally produced by full order models (FOMs) in under‐resolved or marginally‐resolved simulations of convection‐dominated flows. In this article, we investigate the role of numerical stabilization in reduced order models (ROMs) of marginally‐resolved, convection‐dominated incompressible flows. Specifically, we investigate the FOM–ROM consistency, that is, whether the numerical stabilization is beneficial both at the FOM and the ROM level. As a numerical stabilization strategy, we focus on the evolve‐filter‐relax (EFR) regularization algorithm, which centers around spatial filtering. To investigate the FOM‐ROM consistency, we consider two ROM strategies: (i) the EFR‐noEFR, in which the EFR stabilization is used at the FOM level, but not at the ROM level; and (ii) the EFR‐EFR, in which the EFR stabilization is used both at the FOM and at the ROM level. We compare the EFR‐noEFR with the EFR‐EFR in the numerical simulation of a 2D incompressible flow past a circular cylinder in the convection‐dominated, marginally‐resolved regime. We also perform model reduction with respect to both time and Reynolds number. Our numerical investigation shows that the EFR‐EFR is more accurate than the EFR‐noEFR, which suggests that FOM‐ROM consistency is beneficial in convection‐dominated, marginally‐resolved flows.
Birth-and-death Processes in Python: The BirDePy Package

Birth-and-death processes (BDPs) form a class of continuous-time Markov chains that are particularly suited to describing the changes in the size of a population over time. Population-size-dependent BDPs (PSDBDPs) allow the rate at which a population grows to depend on the current population size. The main purpose of our new Python package BirDePy is to provide easy-to-use functions that allow the parameters of discretelyobserved PSDBDPs to be estimated. The package can also be used to estimate parameters of continuously-observed PSDBDPs, simulate sample paths, approximate transition probabilities, and generate forecasts. We describe in detail several methods which have been incorporated into BirDePy to achieve each of these tasks. The usage and effectiveness of the package is demonstrated through a variety of examples of PSDBDPs, as well as case studies involving annual population count data of two endangered bird species.
Adversarial Token Attacks on Vision Transformers

Vision transformers rely on a patch token based self attention mechanism, in contrast to convolutional networks. We investigate fundamental differences between these two families of models, by designing a block sparsity based adversarial token attack. We probe and analyze transformer as well as convolutional models with token attacks of varying patch sizes. We infer that transformer models are more sensitive to token attacks than convolutional models, with ResNets outperforming Transformer models by up to $\sim30\%$ in robust accuracy for single token attacks.
Aura: Privacy-preserving augmentation to improve test set diversity in noise suppression applications

Noise suppression models running in production environments are commonly trained on publicly available datasets. However, this approach leads to regressions in production environments due to the lack of training/testing on representative customer data. Moreover, due to privacy reasons, developers cannot listen to customer content. This ‘ears-off’ situation motivates augmenting existing datasets in a privacy-preserving manner. In this paper, we present Aura , a solution to make existing noise suppression test sets more challenging and diverse while limiting the sampling budget. Aura is ‘ears-off’ because it relies on a feature extractor and a metric of speech quality, DNSMOS P.835, both pre-trained on data obtained from public sources. As an application of Aura , we augment a current benchmark test set in noise suppression by sampling audio ﬁles from a new batch of data of 20K clean speech clips from Librivox mixed with noise clips obtained from AudioSet. Aura makes the existing benchmark test set harder by 100% in DNSMOS P.835, a 26% improvement in Spearman’s rank correlation coefﬁcient (SRCC) compared to random sampling and, identiﬁes 73% out-of-distribution samples to augment the test set.
Rubric-based holistic review: A promising route to equitable graduate admissions in physics

As systematic inequities in higher education and society have been brought to the forefront, graduate programs are interested in increasing the diversity of their applicants and enrollees. Yet, structures in place to evaluate applicants may not support such aims. One potential solution to support those aims is rubric-based holistic review. Starting in 2018, our physics department implemented a rubric-based holistic review process for all applicants to our graduate program. The rubric assessed applicants on 18 metrics covering their grades, test scores, research experiences, noncognitive competencies, and fit with the program. We then compared faculty's ratings of applicants by admission status, sex, and undergraduate program over a three-year period. We find that the rubric scores show statistically significant differences between admitted and non-admitted students as hoped and that statistically significant differences based on sex or undergraduate program aligned with known disparities in GRE scores and service work expectations. Our results then suggest rubric-based holistic review as a possible route to making graduate admissions in physics more equitable.
Optimisation of carbohydrate, lipid and biomass productivity in Tetradesmus obliquus using response surface methodology

Abstract Microalgae are known as an alternative source of biomass energy to replace fossil fuels. The aim of the present study is to optimise the growth conditions and increase the lipid, carbohydrate and biomass productivity of Tetradesmus obliquus UPSI-JRM02 using response surface methodology with Design Expert software. Four variables (nitrogen concentration, temperature, pH, and light intensity) were optimised to enhance T. obliquus productivity. The highest lipid (33 ± 1.2mg/L/day), carbohydrate (51 ± 1.5 mg/L/day) and biomass (115 ± 1.4 mg/L/day) productivity were produced at a nitrogen concentration of 400 mg/L NO3 − at 36 °C with pH of 9.8, and light intensity of 23500 lux. The fatty acid methyl ester contains polyunsaturated fatty acid (C18:2, C18:3) and saturated fatty acid (C16:0). These results demonstrate the potential of T. obliquus to produce promising feedstock for biofuel production.
Cognitively Inspired Learning of Incremental Drifting Concepts

Humans continually expand their learned knowledge to new domains and learn new concepts without any interference with past learned experiences. In contrast, machine learning models perform poorly in a continual learning setting, where input data distribution changes over time. Inspired by the nervous system learning mechanisms, we develop a computational model that enables a deep neural network to learn new concepts and expand its learned knowledge to new domains incrementally in a continual learning setting. We rely on the Parallel Distributed Processing theory to encode abstract concepts in an embedding space in terms of a multimodal distribution. This embedding space is modeled by internal data representations in a hidden network layer. We also leverage the Complementary Learning Systems theory to equip the model with a memory mechanism to overcome catastrophic forgetting through implementing pseudo-rehearsal. Our model can generate pseudo-data points for experience replay and accumulate new experiences to past learned experiences without causing cross-task interference.
Differential Motion Evolution for Fine-Grained Motion Deformation in Unsupervised Image Animation

Image animation is the task of transferring the motion of a driving video to a given object in a source image. While great progress has recently been made in unsupervised motion transfer, requiring no labeled data or domain priors, many current unsupervised approaches still struggle to capture the motion deformations when large motion/view discrepancies occur between the source and driving domains. Under such conditions, there is simply not enough information to capture the motion field properly. We introduce DiME (Differential Motion Evolution), an end-to-end unsupervised motion transfer framework integrating differential refinement for motion estimation. Key findings are twofold: (1) by capturing the motion transfer with an ordinary differential equation (ODE), it helps to regularize the motion field, and (2) by utilizing the source image itself, we are able to inpaint occluded/missing regions arising from large motion changes. Additionally, we also propose a natural extension to the ODE idea, which is that DiME can easily leverage multiple different views of the source object whenever they are available by modeling an ODE per view. Extensive experiments across 9 benchmarks show DiME outperforms the state-of-the-arts by a significant margin and generalizes much better to unseen objects.
Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition

Speech enhancement (SE) aims to suppress the additive noise from noisy speech signals to improve the speech’s perceptual quality and intelligibility. However, the over-suppression phenomenon in the enhanced speech might degrade the performance of downstream automatic speech recognition (ASR) task due to the missing latent information. To alleviate such problem, we propose an interactive feature fusion network (IFF-Net) for noise-robust speech recognition to learn complementary information from the enhanced feature and original noisy feature. Experimental results show that the proposed method achieves absolute word error rate (WER) reduction of 4.1% over the best baseline on RATS Channel-A corpus. Our further analysis indicates that the proposed IFF-Net can complement some missing information in the over-suppressed enhanced feature.
RankingMatch: Delving into Semi-Supervised Learning with Consistency Regularization and Ranking Loss

Semi-supervised learning (SSL) has played an important role in leveraging unlabeled data when labeled data is limited. One of the most successful SSL approaches is based on consistency regularization, which encourages the model to produce unchanged with perturbed input. However, there has been less attention spent on inputs that have the same label. Motivated by the observation that the inputs having the same label should have the similar model outputs, we propose a novel method, RankingMatch, that considers not only the perturbed inputs but also the similarity among the inputs having the same label. We especially introduce a new objective function, dubbed BatchMean Triplet loss, which has the advantage of computational efficiency while taking into account all input samples. Our RankingMatch achieves state-of-the-art performance across many standard SSL benchmarks with a variety of labeled data amounts, including 95.13% accuracy on CIFAR-10 with 250 labels, 77.65% accuracy on CIFAR-100 with 10000 labels, 97.76% accuracy on SVHN with 250 labels, and 97.77% accuracy on SVHN with 1000 labels. We also perform an ablation study to prove the efficacy of the proposed BatchMean Triplet loss against existing versions of Triplet loss.