Miracle-dz/UniFAR-FTUs huggingface dataset:
Neural Rank Collapse: Weight Decay and Small Within-Class Variability Yield Low-Rank Bias

Recent work in deep learning has shown strong empirical and theoretical evidence of an implicit low-rank bias: weight matrices in deep networks tend to be approximately low-rank and removing relatively small singular values during training or from available trained models may significantly reduce model size while maintaining or even improving model performance. However, the majority of the theoretical investigations around low-rank bias in neural networks deal with oversimplified deep linear networks. In this work, we consider general networks with nonlinear activations and the weight decay parameter, and we show the presence of an intriguing neural rank collapse phenomenon, connecting the low-rank bias of trained networks with networks' neural collapse properties: as the weight decay parameter grows, the rank of each layer in the network decreases proportionally to the within-class variability of the hidden-space embeddings of the previous layers. Our theoretical findings are supported by a range of experimental evaluations illustrating the phenomenon.
Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective

Adaptive gradient optimizers like Adam(W) are the default training algorithms for many deep learning architectures, such as transformers. Their diagonal preconditioner is based on the gradient outer product which is incorporated into the parameter update via a square root. While these methods are often motivated as approximate second-order methods, the square root represents a fundamental difference. In this work, we investigate how the behavior of adaptive methods changes when we remove the root, i.e. strengthen their second-order motivation. Surprisingly, we find that such square-root-free adaptive methods close the generalization gap to SGD on convolutional architectures, while maintaining their root-based counterpart's performance on transformers. The second-order perspective also has practical benefits for the development of adaptive methods with non-diagonal preconditioner. In contrast to root-based counterparts like Shampoo, they do not require numerically unstable matrix square roots and therefore work well in low precision, which we demonstrate empirically. This raises important questions regarding the currently overlooked role of adaptivity for the success of adaptive methods since the success is often attributed to sign descent induced by the root.
Late Miocene to Earliest Pliocene (9.84.5 Ma) Paleoceanography of the Subantarctic Southeast Atlantic: Stable Isotopic, Sedimentologic, and Microfossil Evidence

Carbonate stratigraphy and stable isotopic ratios of benthic and planktonic foraminifers were used to study paleoceanographic changes that occurred during the late Miocene to earliest Pliocene in the subantarctic South Atlantic, between 9.8 and 4.5 Ma in ODP Hole 704B on the Meteor Rise (47°S, 7°E; 2532 m water depth). During the late Miocene, between 9.8 and 6.4 Ma, carbonate content was high with little variability (generally 84.5% ± 10%), with sustained productivity dominated by foraminifers and calcareous nannoplankton in surface waters north of the Subantarctic Front. Decreased carbonate (40%), along with first significant occurrence of biogenic opal, occurred between 8.45 and 8.2 Ma. The first signals of increased cooling occurred between 8.8 and 8.0 Ma. The interval from 6.3 to 4.5 Ma represents low carbonate values with high variability (61.7% ± 17%), suggesting markedly fluctuating conditions in the production and/or dissolution of carbonate. The onset of this interval in Hole 704B is marked by a decrease in carbonate values and a well-defined 0.85°/oo decrease in δC values of both planktonic and benthic foraminifers between 6.4 and 6.0 Ma, correlated to the Chron C3AR (upper reversed of Chron 6) "carbon shift. " The interval of the carbon shift (6.4 to 6.0 Ma) is characterized by decreasing δ 1 8 θ values, anomalously low δ 1 8 θ minima in planktonic foraminifers, and intervals dominated by temperate and low-latitude diatom and silicoflagellate assemblages, suggesting warm interglacial conditions with brief events of extreme warming or low salinity in the subantarctic South Atlantic. An anomalously low planktonic δ 1 8 θ minimum at 6.15 Ma in Hole 704B is correlated with the deposition of the laminated organic-rich Neobrunia ooze found in Hole 701 during this leg and the Ethmodiscus ooze drilled at DSDP Site 520 on Leg 73. Suboxic bottom water may have formed in response to a meltwater lid that temporarily halted the production of Antarctic Bottom Water during the rapid deglaciation events that were occurring in West Antarctica. The benthic δ 1 8 θ record displays a strong glacial interval between 5.8 and 5.4 Ma, which is coincident with the time of a major increase in an upwelling diatom assemblage between 6.0 and 5.4 Ma. The earliest Gilbert Chron (5.35 to 4.77 Ma) was marked by an intense carbonate dissolution event and low surface productivity. This dissolution event, recognized globally, ended abruptly near the base of the Thvera Subchron at 4.8 Ma when carbonate values increased to 75.3% ± 5.2%. The low carbonate content and highly variable surface water conditions between 6.3 and 4.8 Ma are time equivalent to the Messinian Stage in the Mediterranean, suggesting a possible link between the Messinian salinity crisis and the Southern Ocean during the latest Miocene.
Measuring Implicit Bias in Explicitly Unbiased Large Language Models

Large language models (LLMs) can pass explicit bias tests but still harbor implicit biases, similar to humans who endorse egalitarian beliefs yet exhibit subtle biases. Measuring such implicit biases can be a challenge: as LLMs become increasingly proprietary, it may not be possible to access their embeddings and apply existing bias measures; furthermore, implicit biases are primarily a concern if they affect the actual decisions that these systems make. We address both of these challenges by introducing two measures of bias inspired by psychology: LLM Implicit Association Test (IAT) Bias, which is a prompt-based method for revealing implicit bias; and LLM Decision Bias for detecting subtle discrimination in decision-making tasks. Using these measures, we found pervasive human-like stereotype biases in 6 LLMs across 4 social domains (race, gender, religion, health) and 21 categories (weapons, guilt, science, career among others). Our prompt-based measure of implicit bias correlates with embedding-based methods but better predicts downstream behaviors measured by LLM Decision Bias. This measure is based on asking the LLM to decide between individuals, motivated by psychological results indicating that relative not absolute evaluations are more related to implicit biases. Using prompt-based measures informed by psychology allows us to effectively expose nuanced biases and subtle discrimination in proprietary LLMs that do not show explicit bias on standard benchmarks.
Reinforcement Learning from Bagged Reward: A Transformer-based Approach for Instance-Level Reward Redistribution

In reinforcement Learning (RL), an instant reward signal is generated for each action of the agent, such that the agent learns to maximize the cumulative reward to obtain the optimal policy. However, in many real-world applications, the instant reward signals are not obtainable by the agent. Instead, the learner only obtains rewards at the ends of bags, where a bag is defined as a partial sequence of a complete trajectory. In this situation, the learner has to face the significant difficulty of exploring the unknown instant rewards in the bags, which could not be addressed by existing approaches, including those trajectory-based approaches that consider only complete trajectories and ignore the inner reward distributions. To formally study this situation, we introduce a novel RL setting termed Reinforcement Learning from Bagged Rewards (RLBR), where only the bagged rewards of sequences can be obtained. We provide the theoretical study to establish the connection between RLBR and standard RL in Markov Decision Processes (MDPs). To effectively explore the reward distributions within the bagged rewards, we propose a Transformer-based reward model, the Reward Bag Transformer (RBT), which uses the self-attention mechanism for interpreting the contextual nuances and temporal dependencies within each bag. Extensive experimental analyses demonstrate the superiority of our method, particularly in its ability to mimic the original MDP's reward distribution, highlighting its proficiency in contextual understanding and adaptability to environmental dynamics.
Large-scale Generative AI Models Lack Visual Number Sense

Humans can readily judge the number of objects in a visual scene, even without counting, and such a skill has been documented in a variety of animal species and in babies prior to language development and formal schooling. Numerical judgments are error-free for small sets, while for larger collections responses become approximate, with variability increasing proportionally to the target number. This response pattern is observed for items of all kinds, despite variation in object features (such as color or shape), suggesting that our visual number sense relies on abstract representations of numerosity. Here, we investigated whether generative Artificial Intelligence (AI) models based on large-scale transformer architectures can reliably name the number of objects in simple visual stimuli or generate images containing a target number of items in the 1-10 range. Surprisingly, none of the foundation models considered performed in a human-like way: They all made striking errors even with small numbers, the response variability often did not increase in a systematic way, and the pattern of errors varied with object category. Our findings demonstrate that advanced AI systems still lack a basic ability that supports an intuitive understanding of numbers, which in humans is foundational for numeracy and mathematical development.
Measurement of $CP$ asymmetries in $B^0\to\eta'K^0_s$ decays at Belle II

We describe a measurement of charge-parity ($CP$) violation asymmetries in $B^0\to\eta'K^0_S$ decays using Belle II data. We consider $\eta'\to\eta(\to\gamma\gamma)\pi^+\pi^-$ and $\eta'\to\rho(\to\pi^+\pi^-)\gamma$ decays. The data were collected at the SuperKEKB asymmetric-energy $e^+e^-$ collider between the years 2019 and 2022, and contain $(387\pm 6) \times 10^6$ bottom-antibottom meson pairs. We reconstruct $829\pm35$ signal decays and extract the $CP$ violating parameters from a fit to the distribution of the proper-decay-time difference between the two $B$ mesons. The measured direct and mixing-induced $CP$ asymmetries are $\text{C}_{\eta'K^0_S} = -0.19 \pm 0.08 \pm 0.03 $ and $\text{S}_{\eta'K^0_S} = +0.67 \pm 0.10 \pm 0.04 $, respectively, where the first uncertainties are statistical and the second are systematic. These results are in agreement with current world averages and standard model predictions.
Tabular Data: Is Attention All You Need?

Deep Learning has revolutionized the field of AI and led to remarkable achievements in applications involving image and text data. Unfortunately, there is inconclusive evidence on the merits of neural networks for structured tabular data. In this paper, we introduce a large-scale empirical study comparing neural networks against gradient-boosted decision trees on tabular data, but also transformer-based architectures against traditional multi-layer perceptrons (MLP) with residual connections. In contrast to prior work, our empirical findings indicate that neural networks are competitive against decision trees. Furthermore, we assess that transformer-based architectures do not outperform simpler variants of traditional MLP architectures on tabular datasets. As a result, this paper helps the research and practitioner communities make informed choices on deploying neural networks on future tabular data applications.
P-ROCKET: Pruning Random Convolution Kernels for Time Series Classification from a Feature Selection Perspective

In recent years, two competitive time series classification models, namely, ROCKET and MINIROCKET, have garnered considerable attention due to their low training cost and high accuracy. However, they require a large number of random 1-D convolutional kernels to comprehensively capture features, which is incompatible with resource-constrained devices. Despite the development of heuristic algorithms designed to recognize and prune redundant kernels, the inherent time-consuming nature of evolutionary algorithms hinders efficient evaluation. To effectively prune models, this paper removes redundant random kernels from a feature selection perspective by eliminating associating connections in the sequential classifier. Two innovative algorithms are proposed, where the first ADMM-based algorithm formulates the pruning challenge as a group elastic net classification problem, and the second core algorithm named P-ROCKET greatly accelerates the first one by bifurcating the problem into two sequential stages. Stage 1 of P-ROCKET introduces dynamically varying penalties to efficiently implement group-level regularization to delete redundant kernels, and Stage 2 employs element-level regularization on the remaining features to refit a linear classifier for better performance. Experimental results on diverse time series datasets show that P-ROCKET prunes up to 60% of kernels without a significant reduction in accuracy and performs 11 times faster than its counterparts. Our code is publicly available at https://github.com/ShaowuChen/P-ROCKET.
Hexagons all the way down: Grid cells as a conformal isometric map of space

The brain’s ability to navigate is often attributed to spatial cells in the hippocampus and entorhinal cortex. Grid cells, found in the entorhinal cortex, are known for their hexagonal spatial activity patterns and are traditionally believed to be the neural basis for path integration. However, recent studies have cast grid cells as a distance-preserving representation. We further investigate this role in a model of grid cells based on a superposition of plane waves. In a module of such grid cells, we optimise their phases to form a conformal isometry (CI) of two-dimensional flat space. With this setup, we demonstrate that a module of at least seven grid cells can achieve a CI, with phases forming a regular hexagonal arrangement. This pattern persists when increasing the number of cells, significantly diverging from a random uniform distribution. In particular, when optimised for CI, the phase distribution becomes distinctly regular and hexagonal, offering a clear experimentally testable prediction. Moreover, grid modules encoding a CI maintain constant energy expenditure across space, providing a new perspective on the role of energy constraints in normative models of grid cells. Finally, we investigate the minimum number of grid cells required for various spatial encoding tasks, including a unique representation of space, the population activity forming a torus, and achieving a CI, where we find that all three are achieved when the module encodes a CI. Our study not only underscores the versatility of grid cells beyond path integration but also highlights the importance of geometric principles in neural representations of space.
The T-cell niche tunes immune function through modulation of the cytoskeleton and TCR-antigen forces

Obesity is a major public health crisis given its rampant growth and association with an increased risk for cancer. Interestingly, patients with obesity tend to have an increased tumor burden and decreased T-cell function. It remains unclear how obesity compromises T-cell mediated immunity. To address this question, we modeled the adipocyte niche using the secretome released from adipocytes as well as the niche of stromal cells and investigated how these factors modulated T-cell function. We found that the secretomes altered antigen-specific T-cell receptor (TCR) triggering and activation. RNA-sequencing analysis identified thousands of gene targets modulated by the secretome including those associated with cytoskeletal regulation and actin polymerization. We next used molecular force probes to show that T-cells exposed to the adipocyte niche display dampened force transmission to the TCR-antigen complex and conversely, stromal cell secreted factors lead to significantly enhanced TCR forces. These results were then validated in diet-induced obese mice. Importantly, secretome-mediated TCR force modulation mirrored the changes in T-cell functional responses in human T-cells using the FDA-approved immunotherapy, blinatumomab. Thus, this work shows that the adipocyte niche contributes to T-cell dysfunction through cytoskeletal modulation and reduces TCR triggering by dampening TCR forces consistent with the mechanosensor model of T-cell activation.
Characterisation of behaviours relevant to apathy syndrome in the aged male rat

Apathy is a complex psychiatric syndrome characterised by motivational deficit, emotional blunting and cognitive changes. It occurs alongside a broad range of neurological disorders, but also occurs in otherwise healthy ageing. Despite its clinical prevalence, apathy does not yet have a designated treatment strategy. Generation of a translational animal model of apathy syndrome would facilitate the development of novel treatments. Given the multidimensional nature of apathy, a model cannot be achieved with a single behavioural test. Using a battery of behavioural tests we investigated whether aged rats exhibit behavioural deficits across different domains relevant to apathy. Using the effort for reward and progressive ratio tasks we found that aged rats (21-27 months) show intact reward motivation. Using the novelty supressed feeding test and position-based object exploration we found aged rats showed increased anxiety-like behaviour inconsistent with emotional blunting. The sucrose preference test and reward learning assay showed intact reward sensitivity and reward-related cognition in aged rats. However, using a bowl-digging version of the probabilistic reversal learning task, we found a deficit in cognitive flexibility in aged rats that did not translate across to a touchscreen version of the task. While these data reveal important changes in cognitive flexibility and anxiety associated with ageing, aged rats do not show deficits across other behavioural domains relevant to apathy. This suggests that aged rats are not a suitable model for age-related apathy syndrome. These findings contrast with previous work in mice, revealing important species differences in behaviours relevant to apathy syndrome in ageing.
Formation of a giant unilocular vacuole via macropinocytosis-like process confers anoikis resistance

Cell survival in metazoans depends on cell attachment to the extracellular matrix (ECM) or to neighboring cells. Loss of such attachment triggers a type of programmed cell death known as anoikis, the acquisition of resistance to which is a key step in cancer development. The mechanisms underlying anoikis resistance remain unclear, however. The intracellular F-actin cytoskeleton plays a key role in sensing the loss of cell-ECM attachment, but how its disruption affects cell fate during such stress is not well understood. Here we reveal a cell survival strategy characterized by formation of a giant unilocular vacuole (GUVac) in the cytoplasm of the cells whose actin cytoskeleton is disrupted during loss of matrix attachment. Time-lapse imaging and electron microscopy showed that large vacuoles with a diameter of >500 nm accumulated early after inhibition of actin polymerization in cells in suspension culture, and that these vacuoles subsequently coalesced to form a GUVac. GUVac formation was found to result from a variation of a macropinocytosis-like process, characterized by the presence of inwardly curved membrane invaginations. This phenomenon relies on both F-actin depolymerization and the recruitment of septin proteins for micron-sized plasma membrane invagination. The vacuole fusion step during GUVac formation requires PI(3)P produced by VPS34 and PI3K-C2α on the surface of vacuoles. Furthermore, its induction after loss of matrix attachment conferred anoikis resistance. Our results thus show that the formation of a previously unrecognized organelle promotes cell survival in the face of altered actin and matrix environments.
Pharmacological Enhancement of Adult Hippocampal Neurogenesis Improves Behavioral Pattern Separation in Young and Aged Mice

BACKGROUND Impairments in behavioral pattern separation (BPS)—the ability to distinguish between similar contexts or experiences—contribute to memory interference and overgeneralization seen in many neuropsychiatric conditions, including depression, anxiety, PTSD, dementia, and age-related cognitive decline. While BPS relies on the dentate gyrus and is sensitive to changes in adult hippocampal neurogenesis (AHN), its significance as a pharmacological target has not been tested. METHODS In this study, we applied a human neural stem cell high-throughput screening cascade to identify compounds that increase human neurogenesis. One compound with a favorable profile, RO6871135, was then tested in BPS in mice. RESULTS Chronic treatment with RO6871135, 7.5 mg/kg increased AHN and improved BPS in a fear discrimination task in both young and aged mice. RO6871135 treatment also lowered innate anxiety-like behavior, which was more apparent in mice exposed to chronic corticosterone. Ablation of AHN by hippocampal irradiation supported a neurogenesis-dependent mechanism for RO6871135-induced improvements in BPS. To identify possible mechanisms of action, in vitro and in vivo kinase inhibition and chemical proteomics assays were performed. These tests indicated that RO6871135 inhibited CDK8, CDK11, CaMK2a, CaMK2b, MAP2K6, and GSK3b. An analog compound also demonstrated high affinity for CDK8, CaMK2a, and GSK3b. CONCLUSIONS These studies demonstrate a method for empirical identification and preclinical testing of novel neurogenic compounds that can improve BPS, and points to possible novel mechanisms that can be interrogated for the development of new therapies to improve specific endophenotypes such as impaired BPS.
Microgroove substrates unveil topography-driven, dynamic 3D nuclear deformations

Navigating complex extracellular environments requires extensive deformation of cells and their nuclei. Nuclear deformations are intricately linked to nuclear structure and mechanical properties, and abnormalities in nuclear mechanics contribute to various diseases including laminopathies and cancer. Most in vitro systems used to study nuclear deformations are typically designed to generate strong whole-cell confinement relevant for specific cell types such as immune or cancer cells. Here, we use microgroove substrates as a model of anisotropic basement membrane topography and we report that adherent cells including endothelial cells and myoblasts exhibit significant 3D (in-plane and out-of-plane) nuclear deformations, with partial to complete penetration into the microgrooves. These deformations are dynamic with nuclei cyclically entering and exiting the microgrooves. AFM measurements show that these deformation cycles are accompanied by transient changes in nuclear mechanical properties. We also show that nuclear penetration into the grooves is principally driven by cell-substrate adhesion, without the need for cytoskeleton-associated forces. Finally, we demonstrate that myoblasts from patients with LMNA mutations exhibit abnormal nuclear deformations which can be rapidly identified and quantified using automated image analysis. We therefore propose the use of microgrooves as a novel simple, tunable, and high throughput system to study nuclear deformations in adherent cells, with the potential to serve as a functional diagnostic platform for pathological alterations in nuclear mechanics.