Miracle-dz/UniFAR-FTUs huggingface dataset:
Intrinsic Biologically Plausible Adversarial Training

Artificial Neural Networks (ANNs) trained with Backpropagation (BP) excel in different daily tasks but have a dangerous vulnerability: inputs with small targeted perturbations, also known as adversarial samples, can drastically disrupt their performance. Adversarial training, a technique in which the training dataset is augmented with exemplary adversarial samples, is proven to mitigate this problem but comes at a high computational cost. In contrast to ANNs, humans are not susceptible to misclassifying these same adversarial samples, so one can postulate that biologically-plausible trained ANNs might be more robust against adversarial attacks. Choosing as a case study the biologically-plausible learning algorithm Present the Error to Perturb the Input To modulate Activity (PEPITA), we investigate this question through a comparative analysis with BP-trained ANNs on various computer vision tasks. We observe that PEPITA has a higher intrinsic adversarial robustness and, when adversarially trained, has a more favorable natural-vs-adversarial performance trade-off since, for the same natural accuracies, PEPITA's adversarial accuracies decrease in average only by 0.26% while BP's decrease by 8.05%.
In search of dispersed memories: Generative diffusion models are associative memory networks

Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological memory. Generative diffusion models are a type of generative machine learning techniques that have shown great performance in many tasks. Like associative memory systems, these networks define a dynamical system that converges to a set of target states. In this work we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks. This equivalence allows us to interpret the supervised training of diffusion models as a synaptic learning process that encodes the associative dynamics of a modern Hopfield network in the weight structure of a deep neural network. Leveraging this connection, we formulate a generalized framework for understanding the formation of long-term memory, where creative generation and memory recall can be seen as parts of a unified continuum.
The refined class number formula for Drinfeld modules

Let $K/k$ be a finite Galois extension of global function fields. Let $E$ be a Drinfeld module over $k$. We state and prove an equivariant refinement of Taelman's analogue of the analytic class number formula for $(E,K/k)$, and derive explicit consequences for the Galois structure of the Taelman class group of $E$ over $K$.
Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks

Pre-training on large-scale datasets and then fine-tuning on downstream tasks have become a standard practice in deep learning. However, pre-training data often contain label noise that may adversely affect the generalization of the model. This paper aims to understand the nature of noise in pre-training datasets and to mitigate its impact on downstream tasks. More specifically, through extensive experiments of supervised pre-training models on synthetic noisy ImageNet-1K and YFCC15M datasets, we demonstrate that while slight noise in pre-training can benefit in-domain (ID) transfer performance, where the training and testing data share the same distribution, it always deteriorates out-of-domain (OOD) performance, where training and testing data distribution are different. We empirically verify that the reason behind is noise in pre-training shapes the feature space differently. We then propose a light-weight black-box tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization on both ID and OOD tasks, considering one may not be able to fully fine-tune or even access the pre-trained models. We conduct practical experiments on popular vision and language models that are pre-trained on noisy data for evaluation of our approach. Our analysis and results show the importance of this interesting and novel research direction, which we term Noisy Model Learning.
A Method for Identification and Ranking of Requirements Sources

Requirements engineering (RE) literature acknowledges the importance of early stakeholder identification. The sources of requirements are many and also constantly changing as the market and business constantly change. Identifying and consulting all stakeholders on the market is impractical; thus many companies utilize indirect data sources, e.g. documents and representatives of larger groups of stakeholders. However, companies often collect irrelevant data or develop their products based on the sub-optimal information sources that may lead to missing market opportunities. We propose a collaborative method for identification and selection of data sources. The method consists of four steps and aims to build consensus between different perspectives in an organization. We demonstrate the use of the method with three industrial case studies. We have presented and statically validated the method to support prioritization of stakeholders for MDRE. Our results show that the method can support the identification and selection of data sources in three ways: (1) by providing systematic steps to identify and prioritize data sources for RE, (2) by highlighting and resolving discrepancies between different perspectives in an organization, and (3) by analyzing the underlying rationale for using certain data sources.
Narrow and Ultranarrow Transitions in Highly Charged Xe Ions as Probes of Fifth Forces.

Optical frequency metrology in atoms and ions can probe hypothetical fifth forces between electrons and neutrons by sensing minute perturbations of the electronic wave function induced by them. A generalized King plot has been proposed to distinguish them from possible standard model effects arising from, e.g., finite nuclear size and electronic correlations. Additional isotopes and transitions are required for this approach. Xenon is an excellent candidate, with seven stable isotopes with zero nuclear spin, however it has no known visible ground-state transitions for high resolution spectroscopy. To address this, we have found and measured twelve magnetic-dipole lines in its highly charged ions and theoretically studied their sensitivity to fifth forces as well as the suppression of spurious higher-order standard model effects. Moreover, we identified at 764.8753(16) nm a E2-type ground-state transition with 500 s excited state lifetime as a potential clock candidate further enhancing our proposed scheme.
MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Driving

Reconstructing 3D vehicles from noisy and sparse partial point clouds is of great significance to autonomous driving. Most existing 3D reconstruction methods cannot be directly applied to this problem because they are elaborately designed to deal with dense inputs with trivial noise. In this work, we propose a novel framework, dubbed MV-DeepSDF, which estimates the optimal Signed Distance Function (SDF) shape representation from multi-sweep point clouds to reconstruct vehicles in the wild. Although there have been some SDF-based implicit modeling methods, they only focus on single-view-based reconstruction, resulting in low fidelity. In contrast, we first analyze multi-sweep consistency and complementarity in the latent feature space and propose to transform the implicit space shape estimation problem into an element-to-set feature extraction problem. Then, we devise a new architecture to extract individual element-level representations and aggregate them to generate a set-level predicted latent code. This set-level latent code is an expression of the optimal 3D shape in the implicit space, and can be subsequently decoded to a continuous SDF of the vehicle. In this way, our approach learns consistent and complementary information among multi-sweeps for 3D vehicle reconstruction. We conduct thorough experiments on two real-world autonomous driving datasets (Waymo and KITTI) to demonstrate the superiority of our approach over state-of-the-art alternative methods both qualitatively and quantitatively.
FPT Approximation of Generalised Hypertree Width for Bounded Intersection Hypergraphs

Generalised hypertree width ($ghw$) is a hypergraph parameter that is central to the tractability of many prominent problems with natural hypergraph structure. Computing $ghw$ of a hypergraph is notoriously hard. The decision version of the problem, checking whether $ghw(H) \leq k$, is paraNP-hard when parameterised by $k$. Furthermore, approximation of $ghw$ is at least as hard as approximation of Set-Cover, which is known to not admit any fpt approximation algorithms. Research in the computation of ghw so far has focused on identifying structural restrictions to hypergraphs -- such as bounds on the size of edge intersections -- that permit XP algorithms for $ghw$. Yet, even under these restrictions that problem has so far evaded any kind of fpt algorithm. In this paper we make the first step towards fpt algorithms for $ghw$ by showing that the parameter can be approximated in fpt time for graphs of bounded edge intersection size. In concrete terms we show that there exists an fpt algorithm, parameterised by $k$ and $d$, that for input hypergraph $H$ with maximal cardinality of edge intersections $d$ and integer $k$ either outputs a tree decomposition with $ghw(H) \leq 4k(k+d+1+)(2k-1)$, or rejects, in which case it is guaranteed that $ghw(H)>k$. Thus, in the special case, of hypergraphs of bounded edge intersection, we obtain an fpt $O(k^3)$-approximation algorithm for $ghw$.
Optimal Nonlinearities Improve Generalization Performance of Random Features

Random feature model with a nonlinear activation function has been shown to perform asymptotically equivalent to a Gaussian model in terms of training and generalization errors. Analysis of the equivalent model reveals an important yet not fully understood role played by the activation function. To address this issue, we study the"parameters"of the equivalent model to achieve improved generalization performance for a given supervised learning problem. We show that acquired parameters from the Gaussian model enable us to define a set of optimal nonlinearities. We provide two example classes from this set, e.g., second-order polynomial and piecewise linear functions. These functions are optimized to improve generalization performance regardless of the actual form. We experiment with regression and classification problems, including synthetic and real (e.g., CIFAR10) data. Our numerical results validate that the optimized nonlinearities achieve better generalization performance than widely-used nonlinear functions such as ReLU. Furthermore, we illustrate that the proposed nonlinearities also mitigate the so-called double descent phenomenon, which is known as the non-monotonic generalization performance regarding the sample size and the model size.
The relationship between Wadati‐Benioff Zone geometry and P, T and B axes of intermediate and deep focus earthquakes

For 865 earthquakes with focal depths exceeding 90 km this paper presents a concise global summary of the distributions of P, T, and B axes of their focal mechanisms with respect to the downdip, along-strike, and normal axes of Wadati-Benioff zones. The focal mechanisms are best double-couple solutions for the centroid moment tensors reported by the Harvard group, with the P, T, and B axes corresponding to the principal axes of the individual moment tensors. We find, as did previous investigators, that events below 300 km have predominantly downdip P axes, with 50% of all P axes of deep focus events lying within 28° of downdip. For events between 90- and 300-km depth there are regional differences, but most regions possess events with nearly downdip P axes or events with nearly downdip T axes. In almost all regions having downdip P or T axes, Bingham statistics show that the greatest concentration of P or T axes lies somewhat below the plane of the Wadati-Benioff zone. For events of all focal depths in nearly all regions, B axes cluster along the strike direction, with 50% occurring within 38° of the strike. From these regional results we define a “typical” focal mechanism as one having the B axis within 30° of the along strike and having either the T or P axis within 30° of the downdip. Such a focal mechanism might occur in response to a stress tensor having its greatest or least principal stresses channeled approximately along the downdip direction of the subducting slab, with the intermediate principal stress lying horizontally within the slab, controlled possibly by slab bending. However, by this criterion, only 29% of the focal mechanisms for deep and intermediate events are typical. Thus the majority of the individual events cannot be explained simply as occurring in response to a downdip and slab-normal greatest and least principal stress system, with a horizontal along-strike intermediate principal stress. A few regions, such as Java and the Marianas, are clearly anomalous in that they possess significantly fewer typical focal mechanisms than do most other regions.
Relational Constraints On Neural Networks Reproduce Human Biases towards Abstract Geometric Regularity

Uniquely among primates, humans possess a remarkable capacity to recognize and manipulate abstract structure in the service of task goals across a broad range of behaviors. One illustration of this is in the visual perception of geometric forms. Studies have shown a uniquely human bias toward geometric regularity, with task performance enhanced for more regular and symmetric forms compared to their geometrically irregular counterparts. Such studies conclude that this behavior implies the existence of discrete symbolic structure in human mental representations, and that replicating such behavior in neural network architectures will require mechanisms for symbolic processing. In this study, we argue that human biases towards geometric regularity can be reproduced in neural networks, without explicitly providing them with symbolic machinery, by augmenting them with an architectural constraint that enables the system to discover and manipulate relational structure. When trained with the appropriate curriculum, this model exhibits human-like biases towards symmetry and regularity in two distinct tasks involving abstract geometric reasoning. Our findings indicate that neural networks, when equipped with the necessary training objectives and architectural elements, can exhibit human-like regularity biases and generalization. This approach provides insights into the neural mechanisms underlying geometric reasoning and offers an alternative to prevailing symbolic"Language of Thought"models in this domain.
Small ideals in polynomial rings and applications

Let $\mathbf{k}$ be a field which is either finite or algebraically closed and let $R = \mathbf{k}[x_1,\ldots,x_n]. $ We prove that any $g_1,\ldots,g_s\in R$ homogeneous of positive degrees $\le d$ are contained in an ideal generated by an $R_t$-sequence of $\le A(d)(s+t)^{B(d)}$ homogeneous polynomials of degree $\le d,$ subject to some restrictions on the characteristic of $\mathbf{k}. $ This yields effective bounds for new cases of Ananyan and Hochster's theorem A in arXiv:1610.09268 on strength and the codimension of the singular locus. It also implies effective bounds when $d$ equals the characteristic of $\mathbf{k}$ for Tao and Ziegler's result in arXiv:1101.1469 on rank and $U^d$ Gowers norms of polynomials over finite fields.
Channelized bottom melting and stability of floating ice shelves

The floating ice shelf in front of Petermann Glacier, in northwest Greenland, experiences massive bottom melting that removes 80% of its ice before calving into the Arctic Ocean. Detailed surveys of the ice shelf reveal the presence of 1–2 km wide, 200–400 m deep, sub‐ice shelf channels, aligned with the flow direction and spaced by 5 km. We attribute their formation to the bottom melting of ice from warm ocean waters underneath. Drilling at the center of one of channel, only 8 m above sea level, confirms the presence of ice‐shelf melt water in the channel. These deep incisions in ice‐shelf thickness imply a vulnerability to mechanical break up and climate warming of ice shelves that has not been considered previously.
Continental Influence on Canadian Cordilleran Terranes From Nd Isotopic Study, and Significance For Crustal Growth Processes

Nd isotopic data are presented for rocks of the terrane assembly that lies inboard of the Stikine terrane in the Canadian Cordillera of British Columbia and Yukon. These are, from most inboard outward: Cassiar, Kootenay, Slide Mountain, Quesnel, and Cache Creek terranes. They are regarded as documenting a transition from terranes whose evolution was closely tied to that of the North American continental margin out to far‐traveled oceanic terranes. The results emphasize sedimentary rocks as indicators of tectonic position of the crustal fragments. Sedimentary rocks of the Cassiar and Kootenay terranes show a strong connection to miogeoclinal sediment sources. Argillites of Pennsylvanian‐Permian age from a paleontologically controlled section in the Slide Mountain terrane are also consistent with sediment sources in the North American miogeocline. Igneous rocks of the Slide Mountain, Quesnel, and Cache Creek terranes show juvenile oceanic or arc origins based on εNd values between +3 and +10, and are essentially identical with published results. Cache Creek and Quesnel terranes also contain sediments with positive εNd values, suggesting a juvenile, ultimately volcanogenic, origin. Both terranes, however, also contain some Triassic and apparently Pennsyl‐vanian‐Permian sedimentary rocks with negative εNd values between –5 and –7, like those of Devonian to Jurassic sedimentary rocks of the North American miogeocline. Possible explanations include proximity to sources of North American terrigenous sediment, expected in Triassic time, or very far‐traveled fine‐grained sediment in the form of hemipelagic clay or eolian dust for older samples. The addition of a continental sedimentary component to wide areas of the Cordillera represents an important point of comparison to Proterozoic orogens, where this component affected the isotopic signatures, but usually cannot be separately identified due to intense reprocessing during orogenesis.
The Flux Operator

Converged computing brings together the best of both worlds for high performance computing (HPC) and cloud-native communities. In fact, the economic impact of cloud-computing, and need for portability, flexibility, and manageability make it not important, but inevitable. Navigating this uncharted territory requires not just innovation in the technology space, but also effort toward collaboration and sharing of ideas. With these goals in mind, this work first tackles the central component of running batch workflows, whether in cloud or HPC: the workload manager. For cloud, Kubernetes has become the de facto tool for this kind of batch orchestration. For HPC, the next-generation HPC workload manager Flux Framework is analogous -- combining fully hierarchical resource management and graph-based scheduling to support intelligent scheduling and job management. Convergence of these managers would mean the implementation of Flux inside of Kubernetes, allowing for hierarchical resource management and scheduling that scales impressively without burdening the Kubernetes scheduler itself. This paper introduces the Flux Operator -- an on-demand HPC workload manager that is easily deployed in Kubernetes. The work here highlights design decisions, mapping of components between environments, experimental features, and shares the results of experiments that compare performance with an equivalent operator in the space, the MPI Operator. Finally, discussion closes with a review of challenges remaining, and hopes for the future for improved technological innovation and collaboration.
Spurious Feature Diversification Improves Out-of-distribution Generalization

Generalization to out-of-distribution (OOD) data is a critical challenge in machine learning. Ensemble-based methods, like weight space ensembles that interpolate model parameters, have been shown to achieve superior OOD performance. However, the underlying mechanism for their effectiveness remains unclear. In this study, we closely examine WiSE-FT, a popular weight space ensemble method that interpolates between a pre-trained and a fine-tuned model. We observe an unexpected phenomenon, in which WiSE-FT successfully corrects many cases where each individual model makes incorrect predictions, which contributes significantly to its OOD effectiveness. To gain further insights, we conduct theoretical analysis in a multi-class setting with a large number of spurious features. Our analysis predicts the above phenomenon and it further shows that ensemble-based models reduce prediction errors in the OOD settings by utilizing a more diverse set of spurious features. Contrary to the conventional wisdom that focuses on learning invariant features for better OOD performance, our findings suggest that incorporating a large number of diverse spurious features weakens their individual contributions, leading to improved overall OOD generalization performance. Empirically we demonstrate the effectiveness of utilizing diverse spurious features on a MultiColorMNIST dataset, and our experimental results are consistent with the theoretical analysis. Building upon the new theoretical insights into the efficacy of ensemble methods, we further identify an issue of WiSE-FT caused by the overconfidence of fine-tuned models in OOD situations. This overconfidence magnifies the fine-tuned model's incorrect prediction, leading to deteriorated OOD ensemble performance. To remedy this problem, we propose a novel method called BAlaNced averaGing (BANG), which significantly enhances the OOD performance of WiSE-FT.
The long-term impact of (un)conditional cash transfers on labour market outcomes in Ecuador

Despite the popularity of conditional cash transfers in low- and middle-income countries, evidence on their long-term effects remains scarce. This paper assesses the impact of the Ecuador's Human Development Grant on the formal sector labour market outcomes of children in eligible households. This grant -- one of the first of its kind -- is characterised by weak enforcement of its eligibility criteria. By means of a regression discontinuity design, we find that this programme increased formal employment rates and labour income around a decade after exposure, thereby curbing the intergenerational transmission of poverty. We discuss possible mediating mechanisms based on findings from previous literature and, in particular, provide evidence on how the programme contributed to persistence in school in the medium run.