Miracle-dz/UniFAR-FTUs huggingface dataset:
White-Box Transformers via Sparse Rate Reduction

In this paper, we contend that the objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a mixture of low-dimensional Gaussian distributions supported on incoherent subspaces. The quality of the final representation can be measured by a unified objective function called sparse rate reduction. From this perspective, popular deep networks such as transformers can be naturally viewed as realizing iterative schemes to optimize this objective incrementally. Particularly, we show that the standard transformer block can be derived from alternating optimization on complementary parts of this objective: the multi-head self-attention operator can be viewed as a gradient descent step to compress the token sets by minimizing their lossy coding rate, and the subsequent multi-layer perceptron can be viewed as attempting to sparsify the representation of the tokens. This leads to a family of white-box transformer-like deep network architectures which are mathematically fully interpretable. Despite their simplicity, experiments show that these networks indeed learn to optimize the designed objective: they compress and sparsify representations of large-scale real-world vision datasets such as ImageNet, and achieve performance very close to thoroughly engineered transformers such as ViT. Code is at \url{https://github.com/Ma-Lab-Berkeley/CRATE}.
A singular value decomposition based generalized finite difference method for fluid solid interaction problems

A hybrid meshfree-Cartesian grid method is proposed for simulating three dimensional fluid-solid interaction (FSI) problems involving rigid bodies with large boundary motions. The rigid body is embedded and enveloped by a cloud of mesh-free nodes, which convect with the motion of the body against a background of Cartesian nodes. Spatial discretization is accomplished by the combination of a Generalized Finite Difference (GFD) method and conventional finite difference (FD) method applied to the meshfree and Cartesian nodes respectively. Error minimization in GFD is carried out by singular value decomposition (SVD). A time-implicit iterative procedure is employed to compute the new/evolving position of the immersed bodies together with the dynamically coupled solutions of the flow field and bodies. The present method is applied to simulate the FSI problems of freely falling bodies in quiescent flow and freely rotating bodies in shear flow. The good agreement with published results validates the ability of the present hybrid meshfree-Cartesian grid scheme for solving FSI problems in 3D.
SOFIA/HAWC+ Far-Infrared Polarimetric Large Area CMZ Exploration Survey. I. General Results from the Pilot Program

We present the first data release of the Far-Infrared Polarimetric Large Area CMZ Exploration (FIREPLACE) survey. The survey was taken using the 214 μm band of the HAWC+ instrument with the SOFIA telescope (19.″6 resolution; 0.7 pc). In this first data release we present dust polarization observations covering a ∼0. °5 region of the Galactic center’s central molecular zone (CMZ), approximately centered on the Sgr B2 complex. We detect ∼25,000 Nyquist-sampled polarization pseudovectors, after applying the standard SOFIA cuts for minimum signal-to-noise ratios in fractional polarization and total intensity of three and 200, respectively. Analysis of the magnetic field orientation suggests a bimodal distribution in the field direction. This bimodal distribution shows enhancements in the distribution of field directions for orientations parallel and perpendicular to the Galactic plane, which are suggestive of a CMZ magnetic field configuration with polodial and torodial components. Furthermore, a detailed analysis of individual clouds included in our survey (i.e., Sgr B2, Sgr B2-NW, Sgr B2-Halo, Sgr B1, and Cloud E/F) shows they have fractional polarization values of 1%–10% at 214 μm, with most of the emission having values <5%. A few of these clouds (i.e., Sgr B2 and Cloud E/F) show relatively low fractional polarization values toward their cores and higher fractional polarization values toward their less dense peripheries. We also observe higher fractional polarization toward compact H ii regions, which could indicate an enhancement in the grain alignment in the dust surrounding these sources.
Task-Agnostic Structured Pruning of Speech Representation Models

Self-supervised pre-trained models such as Wav2vec2, Hubert, and WavLM have been shown to significantly improve many speech tasks. However, their large memory and strong computational requirements hinder their industrial applicability. Structured pruning is a hardware-friendly model compression technique but usually results in a larger loss of accuracy. In this paper, we propose a fine-grained attention head pruning method to compensate for the performance degradation. In addition, we also introduce the straight through estimator into the L0 regularization to further accelerate the pruned model. Experiments on the SUPERB benchmark show that our model can achieve comparable performance to the dense model in multiple tasks and outperforms the Wav2vec 2.0 base model on average, with 72% fewer parameters and 2 times faster inference speed.
One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learning

Offline reinforcement learning (RL) is suitable for safety-critical domains where online exploration is too costly or dangerous. In such safety-critical settings, decision-making should take into consideration the risk of catastrophic outcomes. In other words, decision-making should be risk-sensitive. Previous works on risk in offline RL combine together offline RL techniques, to avoid distributional shift, with risk-sensitive RL algorithms, to achieve risk-sensitivity. In this work, we propose risk-sensitivity as a mechanism to jointly address both of these issues. Our model-based approach is risk-averse to both epistemic and aleatoric uncertainty. Risk-aversion to epistemic uncertainty prevents distributional shift, as areas not covered by the dataset have high epistemic uncertainty. Risk-aversion to aleatoric uncertainty discourages actions that may result in poor outcomes due to environment stochasticity. Our experiments show that our algorithm achieves competitive performance on deterministic benchmarks, and outperforms existing approaches for risk-sensitive objectives in stochastic domains.
Convex and Non-Convex Optimization under Generalized Smoothness

Classical analysis of convex and non-convex optimization methods often requires the Lipshitzness of the gradient, which limits the analysis to functions bounded by quadratics. Recent work relaxed this requirement to a non-uniform smoothness condition with the Hessian norm bounded by an affine function of the gradient norm, and proved convergence in the non-convex setting via gradient clipping, assuming bounded noise. In this paper, we further generalize this non-uniform smoothness condition and develop a simple, yet powerful analysis technique that bounds the gradients along the trajectory, thereby leading to stronger results for both convex and non-convex optimization problems. In particular, we obtain the classical convergence rates for (stochastic) gradient descent and Nesterov's accelerated gradient method in the convex and/or non-convex setting under this general smoothness condition. The new analysis approach does not require gradient clipping and allows heavy-tailed noise with bounded variance in the stochastic setting.
Monte Carlo matrix-product-state approach to the false vacuum decay in the monitored quantum Ising chain

In this work we characterize the false vacuum decay in the ferromagnetic quantum Ising chain with a weak longitudinal field subject to continuous monitoring of the local magnetization. Initializing the system in a metastable state, the false vacuum, we study the competition between coherent dynamics, which tends to create resonant bubbles of the true vacuum, and measurements which induce heating and reduce the amount of quantum correlations. To this end we exploit a numerical approach based on the combination of matrix product states with stochastic quantum trajectories which allows for the simulation of the trajectory-resolved non-equilibrium dynamics of interacting many-body systems in the presence of continuous measurements. We show how the presence of measurements affects the false vacuum decay: At short times the departure from the local minimum is accelerated while at long times the system thermalizes to an infinite-temperature incoherent mixture. For large measurement rates the system enters a quantum Zeno regime. The false vacuum decay and the thermalization physics are characterized in terms of the magnetization, connected correlation function, and the trajectory-resolved entanglement entropy.
Unbiased constrained sampling with Self-Concordant Barrier Hamiltonian Monte Carlo

In this paper, we propose Barrier Hamiltonian Monte Carlo (BHMC), a version of the HMC algorithm which aims at sampling from a Gibbs distribution $\pi$ on a manifold $\mathrm{M}$, endowed with a Hessian metric $\mathfrak{g}$ derived from a self-concordant barrier. Our method relies on Hamiltonian dynamics which comprises $\mathfrak{g}$. Therefore, it incorporates the constraints defining $\mathrm{M}$ and is able to exploit its underlying geometry. However, the corresponding Hamiltonian dynamics is defined via non separable Ordinary Differential Equations (ODEs) in contrast to the Euclidean case. It implies unavoidable bias in existing generalization of HMC to Riemannian manifolds. In this paper, we propose a new filter step, called"involution checking step", to address this problem. This step is implemented in two versions of BHMC, coined continuous BHMC (c-BHMC) and numerical BHMC (n-BHMC) respectively. Our main results establish that these two new algorithms generate reversible Markov chains with respect to $\pi$ and do not suffer from any bias in comparison to previous implementations. Our conclusions are supported by numerical experiments where we consider target distributions defined on polytopes.
Light Coreference Resolution for Russian with Hierarchical Discourse Features

Coreference resolution is the task of identifying and grouping mentions referring to the same real-world entity. Previous neural models have mainly focused on learning span representations and pairwise scores for coreference decisions. However, current methods do not explicitly capture the referential choice in the hierarchical discourse, an important factor in coreference resolution. In this study, we propose a new approach that incorporates rhetorical information into neural coreference resolution models. We collect rhetorical featuresfrom automated discourse parses and examine their impact. As a base model, we implement an end-to-end span-based coreference resolver using a partially fine-tuned multilingual entity-aware language model LUKE. We evaluate our method on the RuCoCo-23 Shared Task for coreference resolution in Russian. Our best model employing rhetorical distance between mentions has ranked 1st on the development set (74.6% F1) and 2nd on the test set (73.3% F1) of the Shared Task¹. We hope that our work will inspire further research on incorporating discourse information in neural coreference resolution models.
Analyzing Credit Risk Model Problems through NLP-Based Clustering and Machine Learning: Insights from Validation Reports

This paper explores the use of clustering methods and machine learning algorithms, including Natural Language Processing (NLP), to identify and classify problems identified in credit risk models through textual information contained in validation reports. Using a unique dataset of 657 findings raised by validation teams in a large international banking group between January 2019 and December 2022. The findings are classified into nine validation dimensions and assigned a severity level by validators using their expert knowledge. The authors use embedding generation for the findings' titles and observations using four different pre-trained models, including"module\_url"from TensorFlow Hub and three models from the SentenceTransformer library, namely"all-mpnet-base-v2","all-MiniLM-L6-v2", and"paraphrase-mpnet-base-v2". The paper uses and compares various clustering methods in grouping findings with similar characteristics, enabling the identification of common problems within each validation dimension and severity. The results of the study show that clustering is an effective approach for identifying and classifying credit risk model problems with accuracy higher than 60\%. The authors also employ machine learning algorithms, including logistic regression and XGBoost, to predict the validation dimension and its severity, achieving an accuracy of 80\% for XGBoost algorithm. Furthermore, the study identifies the top 10 words that predict a validation dimension and severity. Overall, this paper makes a contribution by demonstrating the usefulness of clustering and machine learning for analyzing textual information in validation reports, and providing insights into the types of problems encountered in the development and validation of credit risk models.
Semi-supervised Community Detection via Structural Similarity Metrics

Motivated by social network analysis and network-based recommendation systems, we study a semi-supervised community detection problem in which the objective is to estimate the community label of a new node using the network topology and partially observed community labels of existing nodes. The network is modeled using a degree-corrected stochastic block model, which allows for severe degree heterogeneity and potentially non-assortative communities. We propose an algorithm that computes a `structural similarity metric' between the new node and each of the $K$ communities by aggregating labeled and unlabeled data. The estimated label of the new node corresponds to the value of $k$ that maximizes this similarity metric. Our method is fast and numerically outperforms existing semi-supervised algorithms. Theoretically, we derive explicit bounds for the misclassification error and show the efficiency of our method by comparing it with an ideal classifier. Our findings highlight, to the best of our knowledge, the first semi-supervised community detection algorithm that offers theoretical guarantees.
Spectral gaps and Fourier dimension for self-conformal sets with overlaps

We prove a uniform spectral gap for complex transfer operators near the critical line associated to overlapping $C^2$ iterated function systems on the real line satisfying a Uniform Non-Integrability (UNI) condition. Our work extends that of Naud (2005) on spectral gaps for nonlinear Cantor sets to allow overlaps. The proof builds a new method to reduce the problem of the lack of Markov structure to average contraction of products of random Dolgopyat operators. This approach is inspired by a disintegration technique developed by Algom, the first author and Shmerkin in the study of normal numbers. As a consequence of the method of the second author and Stevens, our spectral gap result implies that the Fourier transform of any non-atomic self-conformal measure decays to zero at a polynomial rate for any $C^{2}$ iterated function system satisfying UNI. This latter result leads to Fractal Uncertainty Principles with arbitrary overlaps.
The $q$-Analogue of Zero Forcing for Certain Families of Graphs

Zero forcing is a combinatorial game played on a graph with the ultimate goal of changing the colour of all the vertices at minimal cost. Originally this game was conceived as a one player game, but later a two-player version was devised in-conjunction with studies on the inertia of a graph, and has become known as the $q$-analogue of zero forcing. In this paper, we study and compute the $q$-analogue zero forcing number for various families of graphs. We begin with by considering a concept of contraction associated with trees. We then significantly generalize an equation between this $q$-analogue of zero forcing and a corresponding nullity parameter for all threshold graphs. We close by studying the $q$-analogue of zero forcing for certain Kneser graphs, and a variety of cartesian products of structured graphs.
Clinical Characteristics of Enterococcus-Associated Peritonitis in Patients with Peritoneal Dialysis

Background This study aims to investigate the clinical characteristics of enterococcus-associated peritonitis in patients with peritoneal dialysis (PD). Methods In this retrospective study, patients with PD-associated enterococcal peritonitis (Group E) who were treated in our center between January 2010 and September 2020 were included. Patients with PD-associated streptococcus peritonitis (Group S) and patients with coagulase-negative staphylococcus peritonitis (Group CNS) were matched 1:1 as cohort-control groups. The clinical characteristics and prognosis of these patients were analyzed. Results A total of 21 peritonitis episodes were noted in nine males and nine females, with an average age of 60.33±14.79 years and an average dialysis duration of 63.56±35.23 months. Mixed infection was observed in 7 out of 21 cases. A total of 22 strains of enterococci were identified in bacterial culture, all sensitive to vancomycin. There were significant differences in white blood cell (WBC) count and blood urea nitrogen (BUN) level among three groups on admission (p<0.05). In all three groups, WBC count on the second and third day post-treatment was higher in Group E than in other groups (p<0.05). The cure rate in Group E was lower than in other groups (p<0.01). The mortality rate in Group E was slightly higher than in other groups (p>0.05). Kaplan–Meier analysis revealed a significant difference in the cumulative survival among three groups (p<0.05). Conclusion Enterococcus peritonitis is a rare and severe complication of peritoneal dialysis. Although vancomycin is effective for the treatment of Enterococcus infection, Enterococcus peritonitis still has a high rate of treatment failure, poor response to treatment, and poor prognosis as compared to CNS and streptococcus-related infections.
Mental Health Prescribers’ Perceptions on the Use of Pharmacogenetic Testing in the Management of Depression in the Middle East and North Africa Region

Objective A wide variety of commercial pharmacogenetic (PGx) tools are available worldwide to guide treatment selection for depression based on individuals’ genetic profiles. However, the use of genetic testing to inform psychiatric care has faced challenges due to the limited training and education for mental health clinicians. The aim of this study was to explore the knowledge, level of engagement, and perspectives on the use of PGx testing when making depression management decisions among practicing psychiatrists within the Middle East and North Africa (MENA) region. Methods This is a qualitative study using semi-structured interviews. Consenting psychiatrists were interviewed through an online platform (SkypeTM or Microsoft TeamsTM). Interviews were audio recorded, transcribed, and thematically analyzed with the assistance of NVivo® software. Results Eighteen interviews from 12 countries have been conducted. Analysis of the current interviews produced five major themes including: (1) Overall perceptions and attitudes; (2) Knowledge and awareness; (3) Education, training, and professional experience; (4) Facilitators and barriers; and (5) Ethical dilemmas. These themes support the notion that there is limited, mostly basic, education, knowledge, and training regarding genetic testing in the management of depression, although there is significant interest and willingness in the part of prescribers to adopt this strategy in their practice. Conclusion The findings of the study suggest that psychiatrists practicing in the MENA region appear to be interested in implementing PGx testing when managing people with depression. However, it is also important to recognize that this cannot be achieved unless more supporting strategies are implemented within their current health system environment.
Will the solar cycle 24 be a low one

A simple and applicable method is proposed for comparing the smoothed monthly mean sunspot numbers. Considering the calculating precision for the national sunspot numbers and the time series characteristics of the existing measured data,the range of the smoothed monthly mean sunspot numbers from January to April in 2009 is estimated according to the measured data and the sunspot numbers comparison equations deducted in this paper. That the solar cycle 24 should start in December 2008 is given. Then the ascending length and the maximum sunspot number are predicted,using the statistical formulae and the conceptions of similar cycle and similar phase which are developed in our previous work. The peak for solar cycle 24 will be in 60-80,which will appear in the period from September 2013 to May 2014 and seems to be much lower than 120.8,the peak of solar cycle 23.
Nutritional ecology of European rabbit (Oryctolagus cuniculus): Factors affecting chemical composition of gastric content.

Nutritional ecology seeks to unravel the extensive web of nutritional links that directs animals in their interactions with their ecological and social environments. European rabbit (Oryctolagus cuniculus) populations its endemic locations are declining and it is considered a keystone species of the Mediterranean ecosystem prompteing the interest in its conservation. The main aim of this study was to determine the nutritional composition of the diet of European rabbits through the relative and absolute chemical composition of the gastric content. To address this objective, gastric content was collected from 80 European rabbits in a Mediterranean area for the analysis of its chemical composition. To this end, gastric content was analyzed for dry matter (DM), organic matter (OM), ash, crude protein (CP), highly digestible nonnitrogenous nutrients (HDNN), neutral detergent fibre (NDF), acid detergent fibre (ADF) and lignin. The rabbits were divided into two groups: EMPTY and FULL, depending on the level of stomach filling, directly related to food intake. Our results revealed a positive correlation between the rabbits weight with DM in the gastric content, total gastric content with DM in the gastric content, and DM in gastric content with all chemical parameters analysed. The mean relative values obtained were 8.8%, 25.5%, 40.4% and 25.4%, for ash, CP, NDF and HDNN, respectively. Moreover, EMPTY rabbits had both a proportional (+19%, p = 0.002 and -40%; p = 0.004, on NDF and HDNN, respectively) and absolute (-38%, p = 0.014, -52%; p = 0.012, -52%; p = 0.011 and +83%; p = 0.008 for OM, ash, HDNN, and lignin, respectively) different proportion of nutrients in gastric contents than FULL animals. Since there is a connection between this availability and the fitness of this species, understanding the chemical composition of the rabbit's diet can be utilised to delve into its biology. Our study provides information that will help elucidate the factors affecting the chemical composition of the gastric content of European rabbits to assist land use planners and conservationists in identifying sites for conservation in Mediterranean ecosystems.