Miracle-dz/UniFAR-FTUs huggingface dataset:
Theory on Forgetting and Generalization of Continual Learning

Continual learning (CL), which aims to learn a sequence of tasks, has attracted significant recent attention. However, most work has focused on the experimental performance of CL, and theoretical studies of CL are still limited. In particular, there is a lack of understanding on what factors are important and how they affect"catastrophic forgetting"and generalization performance. To fill this gap, our theoretical analysis, under overparameterized linear models, provides the first-known explicit form of the expected forgetting and generalization error. Further analysis of such a key result yields a number of theoretical explanations about how overparameterization, task similarity, and task ordering affect both forgetting and generalization error of CL. More interestingly, by conducting experiments on real datasets using deep neural networks (DNNs), we show that some of these insights even go beyond the linear models and can be carried over to practical setups. In particular, we use concrete examples to show that our results not only explain some interesting empirical observations in recent studies, but also motivate better practical algorithm designs of CL.
CONTEMPORARY OPTICAL SPECTRAL CLASSIFICATION OF THE OB STARS: A DIGITAL ATLAS

Some recent developments in the optical classification of OB spectra are reviewed in terms of a comprehensive atlas of new blue-violet digital data from the CTIO 1-meter photon-counting system. These developments include the O3 spectral type; luminosity criteria for the O stars; OBN/OBC anomalies; and refined, interpolated late-O/early-B types. Examples of these phenomena are included among extensive spectral- and luminosity-class sequences, comprising 75 standard objects arranged into 27 montages and covering the wavelength range 3950-4750 A for types O3-B3 (-B8 at Ia). It is intended that this atlas serve a reference function analogous to that of the printed MK atlases, for morphological investigations of OB spectra based on digital data, which will supersede photographic techniques in most future applications.
Ergodicity-to-localization transition on random regular graphs with large connectivity and in many-body quantum dots

Anderson localization on random regular graphs (RRG) serves as a toy-model of many-body localization (MBL). We explore the transition for ergodicity to localization on RRG with large connectivity $m$. In the analytical part, we focus on the inverse participation ratio of eigenstates and identify several regimes on the ergodic side of the transition -- self-consistent golden rule, golden rule, precritical, and critical -- that the system consecutively goes through when the disorder increases towards the point $W_c$ of the localization transition. We also perform exact-diagonalization numerics as well as a population-dynamics analysis that combines analytical and numerical techniques. Results of all the approaches are in excellent mutual agreement. We further explore the evolution from ergodicity to localization in two models of Fock-space MBL: fermionic and spin quantum dots. Large-connectivity RRG models serve as an approximation for this class of many-body problems; one of our central goals is to better understand the status of this approximation. Our numerical simulations support the conjecture that, in the limit of a large system, there is a sharp ergodicity-to-localization transition in the quantum-dot models. Furthermore, our results are consistent the RRG-like scaling of the critical disorder, $W_c \sim m \ln m$. While in the golden-rule range the behavior of quantum-dot models is in agreement with analytical predictions based on RRG model, substantial deviations occur in the pre-critical regime. Our results indicate that the pre-critical and critical behavior (as well as the numerical coefficient in the formula for $W_c$) in the quantum-dot models may be different from what one would expect from the RRG-like approximation.
Efficient Fraud Detection using Deep Boosting Decision Trees

Fraud detection is to identify, monitor, and prevent potentially fraudulent activities from complex data. The recent development and success in AI, especially machine learning, provides a new data-driven way to deal with fraud. From a methodological point of view, machine learning based fraud detection can be divided into two categories, i.e., conventional methods (decision tree, boosting.) and deep learning, both of which have significant limitations in terms of the lack of representation learning ability for the former and interpretability for the latter. Furthermore, due to the rarity of detected fraud cases, the associated data is usually imbalanced, which seriously degrades the performance of classification algorithms. In this paper, we propose deep boosting decision trees (DBDT), a novel approach for fraud detection based on gradient boosting and neural networks. In order to combine the advantages of both conventional methods and deep learning, we first construct soft decision tree (SDT), a decision tree structured model with neural networks as its nodes, and then ensemble SDTs using the idea of gradient boosting. In this way we embed neural networks into gradient boosting to improve its representation learning capability and meanwhile maintain the interpretability. Furthermore, aiming at the rarity of detected fraud cases, in the model training phase we propose a compositional AUC maximization approach to deal with data imbalances at algorithm level. Extensive experiments on several real-life fraud detection datasets show that DBDT can significantly improve the performance and meanwhile maintain good interpretability. Our code is available at https://github.com/freshmanXB/DBDT.
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD

Cyclic and randomized stepsizes are widely used in the deep learning practice and can often outperform standard stepsize choices such as constant stepsize in SGD. Despite their empirical success, not much is currently known about when and why they can theoretically improve the generalization performance. We consider a general class of Markovian stepsizes for learning, which contain i.i. d. random stepsize, cyclic stepsize as well as the constant stepsize as special cases, and motivated by the literature which shows that heaviness of the tails (measured by the so-called"tail-index") in the SGD iterates is correlated with generalization, we study tail-index and provide a number of theoretical results that demonstrate how the tail-index varies on the stepsize scheduling. Our results bring a new understanding of the benefits of cyclic and randomized stepsizes compared to constant stepsize in terms of the tail behavior. We illustrate our theory on linear regression experiments and show through deep learning experiments that Markovian stepsizes can achieve even a heavier tail and be a viable alternative to cyclic and i.i. d. randomized stepsize rules.
Data efficiency and extrapolation trends in neural network interatomic potentials

Recently, key architectural advances have been proposed for neural network interatomic potentials (NNIPs), such as incorporating message-passing networks, equivariance, or many-body expansion terms. Although modern NNIP models exhibit small differences in test accuracy, this metric is still considered the main target when developing new NNIP architectures. In this work, we show how architectural and optimization choices influence the generalization of NNIPs, revealing trends in molecular dynamics (MD) stability, data efficiency, and loss landscapes. Using the 3BPA dataset, we uncover trends in NNIP errors and robustness to noise, showing these metrics are insufficient to predict MD stability in the high-accuracy regime. With a large-scale study on NequIP, MACE, and their optimizers, we show that our metric of loss entropy predicts out-of-distribution error and data efficiency despite being computed only on the training set. This work provides a deep learning justification for probing extrapolation and can inform the development of next-generation NNIPs.
Propagation of weakly advantageous mutations in cancer cell population

Research into somatic mutations in cancer cell DNA and their role in tumour growth and progression between successive stages is crucial for improving our understanding of cancer evolution. Mathematical and computer modelling can provide valuable insights into the scenarios of cancer growth, the roles of somatic mutations, and the types and strengths of evolutionary forces they introduce. Previous studies have developed mathematical models of cancer evolution, incorporating driver and passenger somatic mutations. Driver mutations were assumed to have a strong advantageous effect on the growth of the cancer cell population, while passenger mutations were considered fully neutral or mildly deleterious. However, according to several studies, passenger mutations may have a weakly advantageous effect on tumour growth. In this paper, we develop models of cancer evolution with somatic mutations that introduce a weakly advantageous force to the evolution of cancer cells. The models used in this study can be classified into two categories: deterministic and stochastic. Deterministic models are based on systems of differential equations that balance the average number of cells and mutations during evolution. To verify the results of our deterministic modelling, we use a stochastic model based on the Gillespie algorithm. We compare the predictions of our modelling with some observational data on cancer evolution.
Grey Wolf Optimizer and Whale Optimization Algorithm for Stochastic Inventory Management of Reusable Products in a Two-Level Supply Chain

Product reuse and recovery is an efficient tool that helps companies to simultaneously address economic and environmental dimensions of sustainability. This paper presents a novel problem for stock management of reusable products in a single-vendor, multi-product, multi-retailer network. Several constraints, such as the maximum budget, storage capacity, number of orders, etc., are considered in their stochastic form to establish a more realistic problem. The presented problem is formulated using a nonlinear programming mathematical model. The chance-constrained approach is suggested to deal with the constraints’ uncertainty. Regarding the nonlinearity of the model, grey wolf optimizer (GWO) and whale optimization algorithm (WOA) as two novel metaheuristics are presented as solution approaches, and the sequential quadratic programming (SQP) exact algorithm validates their performance. The parameters of algorithms are calibrated using the Taguchi method for the design of experiments. Extensive analysis is established by solving several numerical results in different sizes and utilizing several comparison measures. Also, the results are compared statistically using proper parametric and non-parametric tests. The analysis of the results shows a significant difference between the algorithms, and GWO has a better performance for solving the presented problem. In addition, both algorithms perform well in searching the solution space, where the GWO and WOA differences with the optimal solution of the SQP algorithm are negligible.
Metaphor Detection with Effective Context Denoising

We propose a novel RoBERTa-based model, RoPPT, which introduces a target-oriented parse tree structure in metaphor detection. Compared to existing models, RoPPT focuses on semantically relevant information and achieves the state-of-the-art on several main metaphor datasets. We also compare our approach against several popular denoising and pruning methods, demonstrating the effectiveness of our approach in context denoising. Our code and dataset can be found at https://github.com/MajiBear000/RoPPT.
Fabrication of a Low-Cost Real-Time Mobile ECG System for Health Monitoring

The electrocardiogram (ECG) signal is one of the most vital signals that can be used to investigate the performance of heart. Based on the ECG graph, we can identify different disorders and diseases. Therefore, monitoring this signal is of great importance. Many electrodes (usually 12) are employed to acquire this signal in clinics and hospitals; therefore, a nurse must install them on the body of the patient to record the signal. In this project, we built a device that can acquire the real-time ECG signal and display it on the mobile screen with the least number of electrodes (three electrodes) without requiring a nurse to install the electrodes using the simplest and cheapest type of ICs and transmitters including STM32F030F4P6 microcontroller, AD620 instrumentation amplifier, TL084 amplifier, TC7660 voltage converter, LM1117 regulator, and HC-05 Bluetooth module. Moreover, the device was designed in a way that could operate using a single 9V battery or power adaptor. First, to amplify the signal and remove a part of its noise, the ECG signal is given to the primary analog circuit. Then, in the digital section, using a microcontroller, the signal is discretized, processed, and finally transmitted to a mobile phone for final processing, information extraction, and displaying. In the mobile-written application, we developed a mathematical algorithm based on Pan-Tompkins algorithm to calculate the heartbeat rate, in which the signal peaks are determined after several processing steps using a threshold. In this regard, the ECG signals of 10 subjects were recorded and analyzed to calculate the optimum threshold. Finally, the power spectral density (PSD) and input referred noise were calculated and plotted to check the output signal quality. Based on the PSD and input referred noise amplitudes, we achieved a signal-to-noise ratio of 50dB.
Conditional Positional Encodings for Vision Transformers

We propose a conditional positional encoding (CPE) scheme for vision Transformers. Unlike previous fixed or learnable positional encodings, which are pre-defined and independent of input tokens, CPE is dynamically generated and conditioned on the local neighborhood of the input tokens. As a result, CPE can easily generalize to the input sequences that are longer than what the model has ever seen during training. Besides, CPE can keep the desired translation-invariance in the image classification task, resulting in improved performance. We implement CPE with a simple Position Encoding Generator (PEG) to get seamlessly incorporated into the current Transformer framework. Built on PEG, we present Conditional Position encoding Vision Transformer (CPVT). We demonstrate that CPVT has visually similar attention maps compared to those with learned positional encodings and delivers outperforming results. Our code is available at https://github.com/Meituan-AutoML/CPVT .
The Spectral Properties of Shocked Two-Component Accretion Flows in the Presence of Synchrotron Emission

Two-component advective flows have Keplerian accretion disks on the equatorial plane that is surrounded by sub-Keplerian transonic flows. In this Letter, we study the spectral properties of these flows when the shocks are present. The shock acceleration produces nonthermal electrons in the postshock region that in turn produce power-law synchrotron radiation. The soft photons generated by the bremsstrahlung and synchrotron processes in the sub-Keplerian flow, as well as the multicolor blackbody emission from the Keplerian disk, are Comptonized by the thermal and nonthermal electrons. By varying Keplerian and sub-Keplerian rates, we are able to reproduce the observed soft and hard states as far as the X-ray region is concerned and "low γ-ray intensity" and "high γ-ray intensity" states as far as the soft γ-ray region is concerned. We also find two pivotal points where the spectra intersect, as is observed in, e.g., Cyg X-1.
Few-shot Reranking for Multi-hop QA via Language Model Prompting

We study few-shot reranking for multi-hop QA (MQA) with open-domain questions. To alleviate the need for a large number of labeled question-document pairs for retriever training, we propose PromptRank, which relies on language model prompting for multi-hop path reranking. PromptRank first constructs an instruction-based prompt that includes a candidate document path and then computes the relevance score between a given question and the path based on the conditional likelihood of the question given the path prompt according to a language model. PromptRank yields strong retrieval performance on HotpotQA with only 128 training examples compared to state-of-the-art methods trained on thousands of examples — 73.6 recall@10 by PromptRank vs. 77.8 by PathRetriever and 77.5 by multi-hop dense retrieval.
THERMODYNAMIC FORMALISM FOR AMENABLE GROUPS AND COUNTABLE STATE SPACES

Given the full shift over a countable state space on a countable amenable group, we develop its thermodynamic formalism. First, we introduce the concept of pressure and, using tiling techniques, prove its existence and further properties, such as an infimum rule. Next, we extend the definitions of different notions of Gibbs measures and prove their existence and equivalence, given some regularity and normalization criteria on the potential. Finally, we provide a family of potentials that nontrivially satisfy the conditions for having this equivalence and a nonempty range of inverse temperatures where uniqueness holds.
LES Prediction of Wall-Pressure Fluctuations and Noise of a Low-Speed Airfoil

This paper discusses the prediction of wall-pressure fluctuations and noise of a low-speed flow past a thin cambered airfoil using large-eddy simulation (LES). The results are compared with experimental measurements made in an open-jet anechoic wind-tunnel at Ecole Centrale de Lyon. To account for the effect of the jet on airfoil loading, a Reynolds-averaged Navier-Stokes calculation is first conducted in the full wind-tunnel configuration, and the mean velocities from this calculation are used to define the boundary conditions for the LES in a smaller domain within the potential core of the jet. The LES flow field is characterized by an attached laminar boundary layer on the pressure side of the airfoil and a transitional and turbulent boundary layer on the suction side, in agreement with experimental observations. An analysis of the unsteady surface pressure field shows reasonable agreement with the experiment in terms of frequency spectra and spanwise coherence in the trailing-edge region. In the nose region, characterized by unsteady separation and transition to turbulence, the wall-pressure fluctuations are highly sensitive to small perturbations and thus diffcult to predict or measure with certainty. The LES, in combination with the Ffowcs Williams and Hall solution to the Lighthill equation, also predicts well the radiated trailing-edge noise. A finite-chord correction is derived and applied to the noise prediction, which is shown to improve the overall agreement with the experimental sound spectrum.
A New Algorithm for Low-Frequency Climate Response

Abstract The low-frequency response to changes in external forcing for the climate system is a fundamental issue. In two recent papers the authors developed a new blended response algorithm for predicting the response of a nonlinear chaotic forced-dissipative system to small changes in external forcing. This new algorithm is based on the fluctuation–dissipation theorem and combines the geometrically exact general response formula using integration of a linear tangent model at short response times and the classical quasi-Gaussian response algorithm at longer response times. This algorithm overcomes the inherent numerical instability in the geometric formula arising because of positive Lyapunov exponents at longer times while removing potentially large systematic errors from the classical quasi-Gaussian approximation at moderate times. Here the new blended method is tested on the low-frequency response for a T21 barotropic truncation on the sphere with realistic topography in two dynamical regimes correspon.
Destabilization of plane Poiseuille flow of insulating liquids by unipolar charge injection

The electrohydrodynamic instabilities of plane Poiseuille flow when subjected to an orthogonal arbitrary unipolar injection of charge has been studied. Appropriate boundary conditions for the perturbed charge density in the space charge limited current regime in the presence of forced flows has been derived. The effect of the injection level on the stability of the flow is analyzed, as well as the role played by the ratio of the hydrodynamic to the true ionic mobility. It is shown that in the low Reynolds number region traverse rolls are destabilized more strongly as the ratio of mobilities decreases.
GENERIC PROPERTIES OF THE EIGENVALUE OF THE LAPLACIAN FOR COMPACT RIEMANNIAN MANIFOLDS

Introduction. In this paper, we discuss generic properties of the eigenvalues of the Laplacian for compact Riemannian manifolds without boundary. Throughout this paper, let M be an arbitrary fixed connected compact C°° manifold of dimension n without boundary, and ^t the set of all C°° Riemannian metrics on M. For g e^t, let Δg be the Laplacian (cf. (2.1)) of (M, g) acting on the space C°°(M) of all C°° real valued functions on M and 0 = λo(flf) < λΛflO ^ x2(g) ^ T°° the eigenvalues of the Laplacian Δg counted with their multiplicities. We regard each eigenvalue Xk(g), k = 0,1, 2, , as a function of g in ^. Let us consider the following problem: "Does each eigenvalue Xk(g) depend continuously on g in ^t with respect to the C°° topology*! " The continuous dependence of the eigenvalues of the Dirichlet problem upon variations of domains is well known (cf. [CH, p. 290]). Variations of coefficients of elliptic differential operators were dealt with by KodairaSpencer [KS] who gave a proof of the continuity of eigenvalues. In this paper, we give a simple proof of the above problem. To answer the above problem, in § 1, we introduce a complete distance p on ^t which gives the C°° topology. Then, in § 2, we assert that each χk(g)f k = 1, 2, , depends continuously on g e ^t with respect to the topology on Λ? induced by the distance p. More precisely, we have
Limiting Behavior of Constraint Minimizers for Inhomogeneous Fractional Schr\"{o}dinger Equations

This paper is devoted to the $L^2$-constraint variational problem \begin{equation*} We study $L^2$-normalized solutions of the following inhomogeneous fractional Schr\"{o}dinger equation \begin{equation*} (-\Delta)^{s} u(x)+V(x)u(x)-a|x|^{-b}|u|^{2\beta^2}u(x)=\mu u(x)\ \ \mbox{in}\ \ \R^{N}. \end{equation*} Here $s\in(\frac{1}{2},1)$, $N>2s$, $a>0$, $0<b<\min\{\frac{N}{2},1\}$, $\beta=\sqrt{\frac{2s-b}{N}}$ and $V(x)\geq 0$ is an external potential. We get $L^2$-normalized solutions of the above equation by solving the associated constrained minimization problem. We prove that there exists a critical value $a^*>0$ such that minimizers exist for $0<a<a^*$, and minimizers do not exist for any $a>a^*$. In the case of $a=a^*$, one can obtain the classification results of the existence and non-existence for constraint minimizers, which are depended strongly on the value of $V(0)$. For $V(0)=0$, the limiting behavior of nonnegative minimizers is also analyzed when $a$ tend to $a^*$ from below.