Miracle-dz/UniFAR-FTUs huggingface dataset:
Nonparametric Identification and Estimation of Earnings Dynamics using a Hidden Markov Model: Evidence from the PSID

This paper presents a hidden Markov model designed to investigate the complex nature of earnings persistence. The proposed model assumes that the residuals of log-earnings consist of a persistent component and a transitory component, both following general Markov processes. Nonparametric identification is achieved through spectral decomposition of linear operators, and a modified stochastic EM algorithm is introduced for model estimation. Applying the framework to the Panel Study of Income Dynamics (PSID) dataset, we find that the earnings process displays nonlinear persistence, conditional skewness, and conditional kurtosis. Additionally, the transitory component is found to possess non-Gaussian properties, resulting in a significantly asymmetric distributional impact when high-earning households face negative shocks or low-earning households encounter positive shocks. Our empirical findings also reveal the presence of ARCH effects in earnings at horizons ranging from 2 to 8 years, further highlighting the complex dynamics of earnings persistence.
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent

Decentralized stochastic gradient descent (D-SGD) allows collaborative learning on massive devices simultaneously without the control of a central server. However, existing theories claim that decentralization invariably undermines generalization. In this paper, we challenge the conventional belief and present a completely new perspective for understanding decentralized learning. We prove that D-SGD implicitly minimizes the loss function of an average-direction Sharpness-aware minimization (SAM) algorithm under general non-convex non-$\beta$-smooth settings. This surprising asymptotic equivalence reveals an intrinsic regularization-optimization trade-off and three advantages of decentralization: (1) there exists a free uncertainty evaluation mechanism in D-SGD to improve posterior estimation; (2) D-SGD exhibits a gradient smoothing effect; and (3) the sharpness regularization effect of D-SGD does not decrease as total batch size increases, which justifies the potential generalization benefit of D-SGD over centralized SGD (C-SGD) in large-batch scenarios.
Auto-Validate by-History: Auto-Program Data Quality Constraints to Validate Recurring Data Pipelines

Data pipelines are widely employed in modern enterprises to power a variety of Machine-Learning (ML) and Business-Intelligence (BI) applications. Crucially, these pipelines are recurring (e.g., daily or hourly) in production settings to keep data updated so that ML models can be re-trained regularly, and BI dashboards refreshed frequently. However, data quality (DQ) issues can often creep into recurring pipelines because of upstream schema and data drift over time. As modern enterprises operate thousands of recurring pipelines, today data engineers have to spend substantial efforts to manually monitor and resolve DQ issues, as part of their DataOps and MLOps practices. Given the high human cost of managing large-scale pipeline operations, it is imperative that we can automate as much as possible. In this work, we propose Auto-Validate-by-History (AVH) that can automatically detect DQ issues in recurring pipelines, leveraging rich statistics from historical executions. We formalize this as an optimization problem, and develop constant-factor approximation algorithms with provable precision guarantees. Extensive evaluations using 2000 production data pipelines at Microsoft demonstrate the effectiveness and efficiency of AVH.
Experimental validation of an inverse method for defect reconstruction in a 2D waveguide model

Defect reconstruction is essential in non-destructive testing and structural health monitoring with guided ultrasonic waves. This paper presents an algorithm for reconstructing notches in steel plates which can be seen as artificial defects representing cracks by comparing measured results with those from a simulation model. The model contains a parameterized notch, and its geometrical parameters are to be reconstructed. While the algorithm is formulated and presented in a generalized form for many different defect types, a special case of guided wave propagation is used to investigate one of the simplest possible simulation models that discretizes only the cross-section of the steel plate. An efficient simulation model of the plate cross-section is obtained by the semi-analytical Scaled Boundary Finite Element Method. The reconstruction algorithm applied is gradient-based, and Algorithmic Differentiation calculates the gradient. The dedicated experimental setup excites nearly plane wave fronts propagating orthogonal to the notch. A scanning Laser Doppler Vibrometer records the velocity field at certain points on the plate surface as input to the reconstruction algorithm. Using two plates with notches of different depths, it is demonstrated that accurate geometry reconstruction is possible.
Variational Gaussian Process Diffusion Processes

Diffusion processes are a class of stochastic differential equations (SDEs) providing a rich family of expressive models that arise naturally in dynamic modelling tasks. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach. We propose an alternative parameterization of the Gaussian variational process using a site-based exponential family description. This allows us to trade a slow inference algorithm with fixed-point iterations for a fast algorithm for convex optimization akin to natural gradient descent, which also provides a better objective for learning model parameters.
Cyclic Coverings of genus 2 curves of Sophie Germain type

We consider cyclic unramified coverings of degree d of irreducible complex smooth genus 2 curves and their corresponding Prym varieties. They provide natural examples of polarized abelian varieties with automorphisms of order d. The rich geometry of the associated Prym map, has been studied in several papers, and the cases d=2, 3, 5, 7 are quite well-understood. Nevertheless, very few is known for higher values of d. In this article we investigate if the covering can be reconstructed from its Prym variety, that is, if the generic Prym Torelli Theorem holds for these coverings. We prove this is so for the so-called Sophie Germain prime numbers, that is, for $d\ge 11$ prime such that $(d-1)/2$ is also prime. We use results of arithmetic nature on $GL_2$-type abelian varieties combined with theta-duality techniques.
Computing Education in the Era of Generative AI

The computing education community has a rich history of pedagogical innovation with many efforts, especially at the introductory level, focused on helping students learn how to program. Recent advances in artificial intelligence have led to large language models that can produce source code from natural language problem descriptions with impressive accuracy. The wide availability of these models and their ease of use is raising urgent questions for educators around the need to adapt their pedagogy as well as prompting broader discussion on the computing curriculum. In this article, we discuss the challenges and the opportunities that such models present to computing educators, with a focus on introductory programming classrooms. We organize this discussion around two foundational articles from the computing education literature that were written around the time that awareness of code generating language models was just beginning to emerge. The first of these (in January 2022) evaluated the performance of code generating models on typical introductory-level programming problems, and the second (in August 2022) explored the quality and novelty of learning resources generated by these models. Now, less than two years since the first article was published, we consider implications for computing education in light of new model capabilities and as lessons emerge from educators incorporating such models into their teaching practice.
Deep Joint Source-Channel Coding for Wireless Image Transmission with Entropy-Aware Adaptive Rate Control

Adaptive rate control for deep joint source and channel coding (JSCC) is considered as an effective approach to transmit sufficient information in scenarios with limited communication resources. We propose a deep JSCC scheme for wireless image transmission with entropy-aware adaptive rate control, using a single deep neural network to support multiple rates and automatically adjust the rate based on the feature maps of the input image and their entropy, as well as the channel conditions. In particular, we maximize the entropy of the feature maps to increase the average information carried by each transmitted symbol during the training. We further decide which feature maps should be activated based on their entropy, which improves the efficiency of the transmitted symbols. We also propose a pruning module to remove less important pixels in the activated feature maps in order to further improve transmission efficiency. The experimental results demonstrate that our proposed scheme learns an effective rate control strategy that reduces the required channel bandwidth while preserving the quality of the reconstructed images.
CDLT: A Dataset with Concept Drift and Long-Tailed Distribution for Fine-Grained Visual Categorization

Data is the foundation for the development of computer vision, and the establishment of datasets plays an important role in advancing the techniques of fine-grained visual categorization~(FGVC). In the existing FGVC datasets used in computer vision, it is generally assumed that each collected instance has fixed characteristics and the distribution of different categories is relatively balanced. In contrast, the real world scenario reveals the fact that the characteristics of instances tend to vary with time and exhibit a long-tailed distribution. Hence, the collected datasets may mislead the optimization of the fine-grained classifiers, resulting in unpleasant performance in real applications. Starting from the real-world conditions and to promote the practical progress of fine-grained visual categorization, we present a Concept Drift and Long-Tailed Distribution dataset. Specifically, the dataset is collected by gathering 11195 images of 250 instances in different species for 47 consecutive months in their natural contexts. The collection process involves dozens of crowd workers for photographing and domain experts for labelling. Extensive baseline experiments using the state-of-the-art fine-grained classification models demonstrate the issues of concept drift and long-tailed distribution existed in the dataset, which require the attention of future researches.
Toeplitz operators on two poly-Bergman-type spaces of the Siegel domain $ D_2 \subset \mathbb{C}^2 $ with continuous nilpotent symbols

We studied Toeplitz operators acting on certain poly-Bergman-type spaces of the Siegel domain $ D_{2} \subset \mathbb{C}^{2} $. Using continuous nilpotent symbols, we described the $ C^* $-algebras generated by such Toeplitz operators. Bounded measurable functions of the form $ \tilde{c}(\zeta) = c(\text{Im}\, \zeta_{1}, \text{Im}\, \zeta_{2} - |\zeta_1|^{2}) $ are called nilpotent symbols. In this work, we considered symbols of the form $ \tilde{a}(\zeta) = a(\text{Im}\, \zeta_1) $ and $ \tilde{b}(\zeta) = b(\text{Im}\, \zeta_2 -|\zeta_1|^{2}) $, where both limits $ \lim\limits_{s\rightarrow 0^+} b(s) $ and $ \lim\limits_{s\rightarrow +\infty} b(s) $ exist, and $ a $ belongs to the set of piece-wise continuous functions on $ \overline{\mathbb{R}} = [-\infty, +\infty] $ and with one-sided limits at $ 0 $. We described certain $ C^* $-algebras generated by such Toeplitz operators that turned out to be isomorphic to subalgebras of $ M_n(\mathbb{C}) \otimes C(\overline{\Pi}) $, where $ \overline{\Pi} = \overline{ \mathbb{R}} \times \overline{ \mathbb{R}}_+ $ and $ \overline{\mathbb{R}}_+ = [0, +\infty] $.
Does Character-level Information Always Improve DRS-based Semantic Parsing?

Even in the era of massive language models, it has been suggested that character-level representations improve the performance of neural models. The state-of-the-art neural semantic parser for Discourse Representation Structures uses character-level representations, improving performance in the four languages (i.e., English, German, Dutch, and Italian) in the Parallel Meaning Bank dataset. However, how and why character-level information improves the parser’s performance remains unclear. This study provides an in-depth analysis of performance changes by order of character sequences. In the experiments, we compare F1-scores by shuffling the order and randomizing character sequences after testing the performance of character-level information. Our results indicate that incorporating character-level information does not improve the performance in English and German. In addition, we find that the parser is not sensitive to correct character order in Dutch. Nevertheless, performance improvements are observed when using character-level information.
ContraBAR: Contrastive Bayes-Adaptive Deep RL

In meta reinforcement learning (meta RL), an agent seeks a Bayes-optimal policy -- the optimal policy when facing an unknown task that is sampled from some known task distribution. Previous approaches tackled this problem by inferring a belief over task parameters, using variational inference methods. Motivated by recent successes of contrastive learning approaches in RL, such as contrastive predictive coding (CPC), we investigate whether contrastive methods can be used for learning Bayes-optimal behavior. We begin by proving that representations learned by CPC are indeed sufficient for Bayes optimality. Based on this observation, we propose a simple meta RL algorithm that uses CPC in lieu of variational belief inference. Our method, ContraBAR, achieves comparable performance to state-of-the-art in domains with state-based observation and circumvents the computational toll of future observation reconstruction, enabling learning in domains with image-based observations. It can also be combined with image augmentations for domain randomization and used seamlessly in both online and offline meta RL settings.
Using Unreliable Pseudo-Labels for Label-Efficient Semantic Segmentation

The crux of label-efficient semantic segmentation is to produce high-quality pseudo-labels to leverage a large amount of unlabeled or weakly labeled data. A common practice is to select the highly confident predictions as the pseudo-ground-truths for each pixel, but it leads to a problem that most pixels may be left unused due to their unreliability. However, we argue that every pixel matters to the model training, even those unreliable and ambiguous pixels. Intuitively, an unreliable prediction may get confused among the top classes, however, it should be confident about the pixel not belonging to the remaining classes. Hence, such a pixel can be convincingly treated as a negative key to those most unlikely categories. Therefore, we develop an effective pipeline to make sufficient use of unlabeled data. Concretely, we separate reliable and unreliable pixels via the entropy of predictions, push each unreliable pixel to a category-wise queue that consists of negative keys, and manage to train the model with all candidate pixels. Considering the training evolution, we adaptively adjust the threshold for the reliable-unreliable partition. Experimental results on various benchmarks and training settings demonstrate the superiority of our approach over the state-of-the-art alternatives.
Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning

Histology imaging is an important tool in medical diagnosis and research, enabling the examination of tissue structure and composition at the microscopic level. Understanding the underlying molecular mechanisms of tissue architecture is critical in uncovering disease mechanisms and developing effective treatments. Gene expression profiling provides insight into the molecular processes underlying tissue architecture, but the process can be time-consuming and expensive. We present BLEEP (Bi-modaL Embedding for Expression Prediction), a bi-modal embedding framework capable of generating spatially resolved gene expression profiles of whole-slide Hematoxylin and eosin (H&E) stained histology images. BLEEP uses contrastive learning to construct a low-dimensional joint embedding space from a reference dataset using paired image and expression profiles at micrometer resolution. With this approach, the gene expression of any query image patch can be imputed using the expression profiles from the reference dataset. We demonstrate BLEEP's effectiveness in gene expression prediction by benchmarking its performance on a human liver tissue dataset captured using the 10x Visium platform, where it achieves significant improvements over existing methods. Our results demonstrate the potential of BLEEP to provide insights into the molecular mechanisms underlying tissue architecture, with important implications in diagnosis and research of various diseases. The proposed approach can significantly reduce the time and cost associated with gene expression profiling, opening up new avenues for high-throughput analysis of histology images for both research and clinical applications.
Interpretable Alzheimer's Disease Classification Via a Contrastive Diffusion Autoencoder

In visual object classification, humans often justify their choices by comparing objects to prototypical examples within that class. We may therefore increase the interpretability of deep learning models by imbuing them with a similar style of reasoning. In this work, we apply this principle by classifying Alzheimer's Disease based on the similarity of images to training examples within the latent space. We use a contrastive loss combined with a diffusion autoencoder backbone, to produce a semantically meaningful latent space, such that neighbouring latents have similar image-level features. We achieve a classification accuracy comparable to black box approaches on a dataset of 2D MRI images, whilst producing human interpretable model explanations. Therefore, this work stands as a contribution to the pertinent development of accurate and interpretable deep learning within medical imaging.
DiffECG: A Versatile Probabilistic Diffusion Model for ECG Signals Synthesis

Within cardiovascular disease detection using deep learning applied to ECG signals, the complexities of handling physiological signals have sparked growing interest in leveraging deep generative models for effective data augmentation. In this paper, we introduce a novel versatile approach based on denoising diffusion probabilistic models for ECG synthesis, addressing three scenarios: (i) heartbeat generation, (ii) partial signal imputation, and (iii) full heartbeat forecasting. Our approach presents the first generalized conditional approach for ECG synthesis, and our experimental results demonstrate its effectiveness for various ECG-related tasks. Moreover, we show that our approach outperforms other state-of-the-art ECG generative models and can enhance the performance of state-of-the-art classifiers.
Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RL

Reinforcement learning agents may sometimes develop habits that are effective only when specific policies are followed. After an initial exploration phase in which agents try out different actions, they eventually converge toward a particular policy. When this occurs, the distribution of state-action trajectories becomes narrower, and agents start experiencing the same transitions again and again. At this point, spurious correlations may arise. Agents may then pick up on these correlations and learn state representations that do not generalize beyond the agent's trajectory distribution. In this paper, we provide a mathematical characterization of this phenomenon, which we refer to as policy confounding, and show, through a series of examples, when and how it occurs in practice.
Ashkin-Teller phase transition and multicritical behavior in a classical monomer-dimer model

We use Monte Carlo simulations and tensor network methods to study a classical monomer-dimer model on the square lattice with a hole (monomer) fugacity $z$, an aligning dimer-dimer interaction $u$ that favors columnar order, and an attractive dimer-dimer interaction $v$ between two adjacent dimers that lie on the same principal axis of the lattice. The Monte Carlo simulations of finite size systems rely on our grand-canonical generalization of the dimer worm algorithm, while the tensor network computations are based on a uniform matrix product ansatz for the eigenvector of the row-to-row transfer matrix that work directly in the thermodynamic limit. The phase diagram has nematic, columnar order and fluid phases, and a nonzero temperature multicritical point at which all three meet. For any fixed $v/u<\infty$, we argue that this multicritical point continues to be located at a nonzero hole fugacity $z_{\rm mc}(v/u)>0$; our numerical results confirm this theoretical expectation but find that $z_{\rm mc}(v/u) \to 0$ very rapidly as $v/u \to \infty$. Our numerical results also confirm the theoretical expectation that the corresponding multicritical behavior is in the universality class of the four-state Potts multicritical point on critical line of the two-dimensional Ashkin-Teller model.
Reynolds number asymptotics of wall-turbulence fluctuations

Abstract In continuation of our earlier work (Chen & Sreenivasan, J. Fluid Mech., vol. 908, 2021, R3; Chen & Sreenivasan, J. Fluid Mech., vol. 933, 2022a, A20 – together referred to as CS hereafter), we present a self-consistent Reynolds number asymptotics for wall-normal profiles of variances of streamwise and spanwise velocity fluctuations as well as root-mean-square pressure, across the entire flow region of channel and pipe flows and flat-plate boundary layers. It is first shown that, when normalized by peak values, the Reynolds number dependence and wall-normal variation of all three profiles can be decoupled, in excellent agreement with available data, sharing the common inner expansion of the type $\phi (y^+)=f_0(y^+)+f_1(y^+)/Re^{1/4}_\tau$, where $\phi$ is one of the quantities just mentioned, the functions $f_0$ and $f_1$ depend only on $y^+$, and $Re_\tau$ is the friction Reynolds number. Here, the superscript $+$ indicates normalization by wall variables. We show that this result is completely consistent with CS. Secondly, by matching the above inner expansion and the outer flow similarity form, a bounded variation $\phi (y^\ast )=\alpha _\phi -\beta _{\phi }y^{{\ast {1}/{4}}}$ is derived for the outer region where, for each $\phi$, the constants $\alpha _\phi$ and $\beta _{\phi }$ are independent of $Re_\tau$ and $y^\ast$ $\equiv y^+/Re_\tau$ – also in excellent agreement with simulations and experimental data. One of the predictions of the analysis is that, for asymptotically high Reynolds numbers, a finite plateau $\phi \approx \alpha _\phi$ appears in the outer region. This result sheds light on the intriguing issue of the outer shoulder of the variance of the streamwise velocity fluctuation, which should be bounded by the asymptotic plateau of approximately 10.