Miracle-dz/UniFAR-FTUs huggingface dataset:
Solvable neural network model for input-output associations: Optimal recall at the onset of chaos

In neural information processing, an input modulates neural dynamics to generate a desired output. To unravel the dynamics and underlying neural connectivity enabling such input-output association, we proposed an exactly soluble neural-network model with a connectivity matrix explicitly consisting of inputs and required outputs. An analytic form of the response upon the input is derived, whereas three distinctive types of responses including chaotic dynamics as bifurcation against input strength are obtained depending on the neural sensitivity and number of inputs. Optimal performance is achieved at the onset of chaos, and the relevance of the results to cognitive dynamics is discussed.
Fluctuation-dominated Phase Ordering

Fluctuation-dominated phase ordering refers to a steady state in which the magnitude of long-range order varies strongly owing to fluctuations, and to the associated coarsening phenomena during the approach to steady state. Strong fluctuations can lead to a number of interesting phenomena, including a cusp singularity in the scaled correlation function, implying the breakdown of the Porod Law. First identified in a nonequilibrium system of passively sliding particles on a fluctuating surface, fluctuation-dominated order also occurs in several other systems, including an equilibrium Ising model with long-range interactions. This article discusses these systems, and others where clustering effects are stronger.
Modelling of plasma-wall interaction and impurity transport in fusion devices and prompt deposition of tungsten as application

Main processes of plasma-wall interaction and impurity transport in fusion devices and their impact on the availability of the devices are presented and modelling tools, in particular the three-dimensional Monte-Carlo code ERO, are introduced. The capability of ERO is demonstrated on the example of tungsten erosion and deposition modelling. The dependence of tungsten deposition on plasma temperature and density is studied by simulations with a simplified geometry assuming (almost) constant plasma parameters. The amount of deposition increases with increasing electron temperature and density. Up to 100% of eroded tungsten can be promptly deposited near to the location of erosion at very high densities (∼1 × 1014 cm−3 expected e.g. in the divertor of ITER). The effect of the sheath characteristics on tungsten prompt deposition is investigated by using particle-in-cell (PIC) simulations to spatially resolve the plasma parameters inside the sheath. Applying PIC data instead of non-resolved sheath leads in general to smaller tungsten deposition, which is mainly due to a density and temperature decrease towards the surface within the sheath. Two-dimensional tungsten erosion/deposition simulations, assuming symmetry in toroidal direction but poloidally spatially varying plasma parameter profiles, have been carried out for the JET divertor. The simulations reveal, similar to experimental findings, that tungsten gross erosion is dominated in H-mode plasmas by the intra-ELM phases. However, due to deposition, the net tungsten erosion can be similar within intra- and inter-ELM phases if the inter-ELM electron temperature is high enough. Also, the simulated deposition fraction of about 84% in between ELMs is in line with spectroscopic observations from which a lower limit of 50% has been estimated.
Fluid dynamics from the Boltzmann equation using a maximum entropy distribution

Using the recently developed ``Maximum Entropy'' (or ``least biased'') distribution function to truncate the moment hierarchy arising from kinetic theory, we formulate a far-from-equilibrium macroscopic theory that provides the possibility of describing both free-streaming and hydrodynamic regimes of heavy-ion collisions within a single framework. Unlike traditional hydrodynamic theories that include viscous corrections to finite order, the present formulation incorporates contributions to all orders in shear and bulk inverse Reynolds numbers, allowing it to handle large dissipative fluxes. By considering flow profiles relevant for heavy-ion collisions (Bjorken and Gubser flows), we demonstrate that the present approach provides excellent agreement with underlying kinetic theory throughout the fluid's evolution and, especially, in far-off-equilibrium regimes where traditional hydrodynamics breaks down.
Forking and invariant measures in NIP theories

We give an example of an NIP theory $T$ in which there is a formula that does not fork over $\varnothing$ but has measure $0$ under any global $\varnothing$-invariant Keisler measure, and we show that this cannot occur if $T$ is also first-order amenable.
Optimizing PatchCore for Few/many-shot Anomaly Detection

Few-shot anomaly detection (AD) is an emerging sub-field of general AD, and tries to distinguish between normal and anomalous data using only few selected samples. While newly proposed few-shot AD methods do compare against pre-existing algorithms developed for the full-shot domain as baselines, they do not dedicatedly optimize them for the few-shot setting. It thus remains unclear if the performance of such pre-existing algorithms can be further improved. We address said question in this work. Specifically, we present a study on the AD/anomaly segmentation (AS) performance of PatchCore, the current state-of-the-art full-shot AD/AS algorithm, in both the few-shot and the many-shot settings. We hypothesize that further performance improvements can be realized by (I) optimizing its various hyperparameters, and by (II) transferring techniques known to improve few-shot supervised learning to the AD domain. Exhaustive experiments on the public VisA and MVTec AD datasets reveal that (I) significant performance improvements can be realized by optimizing hyperparameters such as the underlying feature extractor, and that (II) image-level augmentations can, but are not guaranteed, to improve performance. Based on these findings, we achieve a new state of the art in few-shot AD on VisA, further demonstrating the merit of adapting pre-existing AD/AS methods to the few-shot setting. Last, we identify the investigation of feature extractors with a strong inductive bias as a potential future research direction for (few-shot) AD/AS.
Numerical Heat Transfer, Part B: Fundamentals

ISSN: 1040-7790 (Print) 1521-0626 (Online) Journal homepage: http://www.tandfonline. com/loi/unhb20 NEW FAMILY OF ADAPTIVE VERY HIGH RESOLUTION SCHEMES F. Moukalled & M. S. Darwish To cite this article: F. Moukalled & M. S. Darwish (1998) NEW FAMILY OF ADAPTIVE VERY HIGH RESOLUTION SCHEMES, Numerical Heat Transfer, Part B: Fundamentals, 34:2, 215-239 To link to this article: http://dx.doi. org/10.1080/10407799808915055
The Long-Term Effects of Teachers' Gender Stereotypes

This paper studies the effects of teachers' stereotypical assessments of boys and girls on students' long-term outcomes, including high school graduation, college attendance, and formal sector employment. I measure teachers' gender stereotypical grading based on preconceptions about boys' aptitude in math and science and girls' aptitude in communicating and verbal skills by analyzing novel data on teachers' responses to the Implicit Association Test (IAT) and differences in gender gaps between teacher-assigned and blindly graded tests. To collect IAT scores on a national scale, I developed a large-scale educational program accessible to teachers and students in Peruvian public schools. This analysis provides evidence that teachers' gender stereotypes are reflected in student evaluations: math teachers with stronger stereotypes associating boys with scientific disciplines award male students with higher test scores compared to blindly-graded test scores. In contrast, language arts teachers who stereotypically associate females with humanities-based disciplines give female students higher grades. Using graduation, college enrollment, and matched employer-employee data on 1.7 million public high school students expected to graduate between 2015 and 2019, I find that female students who are placed with teachers who use more stereotypical grading practices are less likely to graduate from high school and apply to college than male students. In comparison to their male counterparts, female high school graduates whose teachers employ more stereotypical grading practices are less likely to be employed in the formal sector and have fewer paid working hours. Furthermore, exposure to teachers with more stereotypical grading practices reduces women's monthly earnings, thereby widening the gender pay gap.
Boosting Federated Learning Convergence with Prototype Regularization

As a distributed machine learning technique, federated learning (FL) requires clients to collaboratively train a shared model with an edge server without leaking their local data. However, the heterogeneous data distribution among clients often leads to a decrease in model performance. To tackle this issue, this paper introduces a prototype-based regularization strategy to address the heterogeneity in the data distribution. Specifically, the regularization process involves the server aggregating local prototypes from distributed clients to generate a global prototype, which is then sent back to the individual clients to guide their local training. The experimental results on MNIST and Fashion-MNIST show that our proposal achieves improvements of 3.3% and 8.9% in average test accuracy, respectively, compared to the most popular baseline FedAvg. Furthermore, our approach has a fast convergence rate in heterogeneous settings.
Eliminating Label Leakage in Tree-Based Vertical Federated Learning

Vertical federated learning (VFL) enables multiple parties with disjoint features of a common user set to train a machine learning model without sharing their private data. Tree-based models have become prevalent in VFL due to their interpretability and efficiency. However, the vulnerability of tree-based VFL has not been sufficiently investigated. In this study, we first introduce a novel label inference attack, ID2Graph, which utilizes the sets of record IDs assigned to each node (i.e., instance space)to deduce private training labels. ID2Graph attack generates a graph structure from training samples, extracts communities from the graph, and clusters the local dataset using community information. To counteract label leakage from the instance space, we propose two effective defense mechanisms, Grafting-LDP, which improves the utility of label differential privacy with post-processing, and andID-LMID, which focuses on mutual information regularization. Comprehensive experiments on various datasets reveal that ID2Graph presents significant risks to tree-based models such as RandomForest and XGBoost. Further evaluations of these benchmarks demonstrate that our defense methods effectively mitigate label leakage in such instances
Improving Multimodal Datasets with Image Captioning

Massive web datasets play a key role in the success of large vision-language models like CLIP and Flamingo. However, the raw web data is noisy, and existing filtering methods to reduce noise often come at the expense of data diversity. Our work focuses on caption quality as one major source of noise, and studies how generated captions can increase the utility of web-scraped datapoints with nondescript text. Through exploring different mixing strategies for raw and generated captions, we outperform the best filtering method proposed by the DataComp benchmark by 2% on ImageNet and 4% on average across 38 tasks, given a candidate pool of 128M image-text pairs. Our best approach is also 2x better at Flickr and MS-COCO retrieval. We then analyze what makes synthetic captions an effective source of text supervision. In experimenting with different image captioning models, we also demonstrate that the performance of a model on standard image captioning benchmarks (e.g., NoCaps CIDEr) is not a reliable indicator of the utility of the captions it generates for multimodal training. Finally, our experiments with using generated captions at DataComp's large scale (1.28B image-text pairs) offer insights into the limitations of synthetic text, as well as the importance of image curation with increasing training data quantity. The synthetic captions used in our experiments are now available on HuggingFace.
Period-like polynomials for $L$-series associated with half-integral weight cusp forms

Given the L-series of a half-integral weight cusp form, we construct a cohomology class with coefficients in a finite dimensional vector space in a way that parallels the Eichler cohomology in the integral weight case. We also define a lift of half-integral weight cusp forms to integral weight cusp forms that is compatible with the $L$-series of the respective forms.
Estimation of turbulent channel flow at $Re_{\unicode[STIX]{x1D70F}}~=~100$ based on the wall measurement using a simple sequential approach

The unsteady flow estimation problem of wall-bounded turbulence, numerically benchmarked by Chevalier et al. (J. Fluid Mech., vol. 552, 2006, pp. 167–187), is re-tackled with simple approaches. A turbulent channel flow at $Re_{\unicode[STIX]{x1D70F}}=100$ with periodic boundary conditions is reconstructed with linear stochastic estimation only based on the wall measurement, i.e. the wall shear stress in the streamwise and spanwise directions as well as the wall pressure over the entire wavenumber space. The results reveal that instantaneous information on the wall governs the success of the estimation in the vicinity of the wall ( $y^{+}\lesssim 20$ ). The degrees of agreement are equivalent to those reported by Chevalier et al. using the extended Kalman filter as well as the ensemble Kalman filter performed in this study. This suggests that the instantaneous information on the wall dictates the reconstruction rather than the prediction step in these state observers solving the dynamical system. Subsequently, we feed the velocity components given by the linear stochastic estimation via the body-force term into the Navier–Stokes system: such an observer slightly improves the estimation in the log layer, indicating a small benefit of involving a dynamical system but over-suppression of turbulent motions beyond the viscous sublayer due to their low correlation with the wall measurement. Errors in the estimation grow in the buffer layer and prevent further reconstruction toward the centreline even if we relax the feedback forcing and let the flow evolve nonlinearly through the observer. We also discuss the flow components truly reconstructible from the wall measurement, which has limited degrees of freedom and poor correlation across wavenumbers.
Dynamical tidal response of Kerr black holes from scattering amplitudes

We match scattering amplitudes in point particle effective field theory (EFT) and general relativity to extract low frequency dynamical tidal responses of rotating (Kerr) black holes to all orders in spin. In the conservative sector, we study local worldline couplings that correspond to the time-derivative expansion of the black hole tidal response function. These are dynamical (frequency-dependent) generalizations of the static Love numbers. We identify and extract couplings of three types of subleading local worldline operators: the curvature time derivative terms, the spin - curvature time derivative couplings, and quadrupole - octupole mixing operators that arise due to the violation of spherical symmetry. The first two subleading couplings are non-zero and exhibit a classical renormalization group running; we explicitly present their scheme-independent beta functions. The conservative mixing terms, however, vanish as a consequence of vanishing static Love numbers. In the non-conservative sector, we match the dissipation numbers at next-to-leading and next-to-next-to leading orders in frequency. In passing, we identify terms in the general relativity absorption probabilities that originate from tails and short-scale logarithmic corrections to the lowest order dissipation contributions.
Fluctuation Moments for Regular Functions of Wigner Matrices

We compute the deterministic approximation for mixed fluctuation moments of products of deterministic matrices and general Sobolev functions of Wigner matrices. Restricting to polynomials, our formulas reproduce recent results of [Male, Mingo, Pech\'e, Speicher 2022], showing that the underlying combinatorics of non-crossing partitions and annular non-crossing permutations continue to stay valid beyond the setting of second-order free probability theory. The formulas obtained further characterize the variance in the functional central limit theorem obtained recently in the companion paper [Reker 2023].
Learning Neural Programs To Parse Programs

In recent years, deep learning techniques have been developed to improve the performance of program synthesis from input-output examples. Albeit its significant progress, the programs that can be synthesized by state-of-the-art approaches are still simple in terms of their complexity. In this work, we move a significant step forward along this direction by proposing a new class of challenging tasks in the domain of program synthesis from input-output examples: learning a context-free parser from pairs of input programs and their parse trees. We show that this class of tasks are much more challenging than previously studied tasks, and the test accuracy of existing approaches is almost 0%. We tackle the challenges by developing three novel techniques inspired by three novel observations, which reveal the key ingredients of using deep learning to synthesize a complex program. First, the use of a non-differentiable machine is the key to effectively restrict the search space. Thus our proposed approach learns a neural program operating a domain-specific non-differentiable machine. Second, recursion is the key to achieve generalizability. Thus, we bake-in the notion of recursion in the design of our non-differentiable machine. Third, reinforcement learning is the key to learn how to operate the non-differentiable machine, but it is also hard to train the model effectively with existing reinforcement learning algorithms from a cold boot. We develop a novel two-phase reinforcement learning-based search algorithm to overcome this issue. In our evaluation, we show that using our novel approach, neural parsing programs can be learned to achieve 100% test accuracy on test inputs that are 500x longer than the training samples.
Gradient-Semantic Compensation for Incremental Semantic Segmentation

Incremental semantic segmentation focuses on continually learning the segmentation of new coming classes without obtaining the training data from previously seen classes. However, most current methods fail to tackle catastrophic forgetting and background shift since they 1) treat all previous classes equally without considering different forgetting paces caused by imbalanced gradient back-propagation; 2) lack strong semantic guidance between classes. In this paper, to solve the aforementioned challenges, we propose a Gradient-Semantic Compensation (GSC) model, which surmounts incremental semantic segmentation from both gradient and semantic perspectives. Specifically, to handle catastrophic forgetting from the gradient aspect, we develop a step-aware gradient compensation that can balance forgetting paces of previously seen classes by re-weighting gradient back-propagation. Meanwhile, we propose a soft-sharp semantic relation distillation to distill consistent inter-class semantic relations via soft labels for alleviating catastrophic forgetting from the semantic aspect. In addition, we design a prototypical pseudo re-labeling which provides strong semantic guidance to mitigate background shift. It produces high-quality pseudo labels for background pixels belonging to previous classes by assessing distances of pixels relative to class-wise prototypes. Experiments on three public segmentation datasets provide strong evidence for the effectiveness of our proposed GSC model.
Lasso and elastic nets by orthants

We propose a new method for computing the lasso path, using the fact that the Manhattan norm of the coefficient vector is linear over every orthant of the parameter space. We use simple calculus and present an algorithm in which the lasso path is series of orthant moves. Our proposal gives the same results as standard literature, with the advantage of neat interpretation of results and explicit lasso formul{\ae}. We extend this proposal to elastic nets and obtain explicit, exact formul{\ae} for the elastic net path, and with a simple change, our lasso algorithm can be used for elastic nets. We present computational examples and provide simple R prototype code.
Learning Dense UV Completion for Human Mesh Recovery

Human mesh reconstruction from a single image is challenging in the presence of occlusion, which can be caused by self, objects, or other humans. Existing methods either fail to separate human features accurately or lack proper supervision for feature completion. In this paper, we propose Dense Inpainting Human Mesh Recovery (DIMR), a two-stage method that leverages dense correspondence maps to handle occlusion. Our method utilizes a dense correspondence map to separate visible human features and completes human features on a structured UV map dense human with an attention-based feature completion module. We also design a feature inpainting training procedure that guides the network to learn from unoccluded features. We evaluate our method on several datasets and demonstrate its superior performance under heavily occluded scenarios compared to other methods. Extensive experiments show that our method obviously outperforms prior SOTA methods on heavily occluded images and achieves comparable results on the standard benchmarks (3DPW).
Revisiting Fine-Tuning Strategies for Self-supervised Medical Imaging Analysis

Despite the rapid progress in self-supervised learning (SSL), end-to-end fine-tuning still remains the dominant fine-tuning strategy for medical imaging analysis. However, it remains unclear whether this approach is truly optimal for effectively utilizing the pre-trained knowledge, especially considering the diverse categories of SSL that capture different types of features. In this paper, we present the first comprehensive study that discovers effective fine-tuning strategies for self-supervised learning in medical imaging. After developing strong contrastive and restorative SSL baselines that outperform SOTA methods across four diverse downstream tasks, we conduct an extensive fine-tuning analysis across multiple pre-training and fine-tuning datasets, as well as various fine-tuning dataset sizes. Contrary to the conventional wisdom of fine-tuning only the last few layers of a pre-trained network, we show that fine-tuning intermediate layers is more effective, with fine-tuning the second quarter (25-50%) of the network being optimal for contrastive SSL whereas fine-tuning the third quarter (50-75%) of the network being optimal for restorative SSL. Compared to the de-facto standard of end-to-end fine-tuning, our best fine-tuning strategy, which fine-tunes a shallower network consisting of the first three quarters (0-75%) of the pre-trained network, yields improvements of as much as 5.48%. Additionally, using these insights, we propose a simple yet effective method to leverage the complementary strengths of multiple SSL models, resulting in enhancements of up to 3.57% compared to using the best model alone. Hence, our fine-tuning strategies not only enhance the performance of individual SSL models, but also enable effective utilization of the complementary strengths offered by multiple SSL models, leading to significant improvements in self-supervised medical imaging analysis.