Miracle-dz/UniFAR-FTUs huggingface dataset:
LU decomposition and Toeplitz decomposition of a neural network

It is well-known that any matrix $A$ has an LU decomposition. Less well-known is the fact that it has a 'Toeplitz decomposition' $A = T_1 T_2 \cdots T_r$ where $T_i$'s are Toeplitz matrices. We will prove that any continuous function $f : \mathbb{R}^n \to \mathbb{R}^m$ has an approximation to arbitrary accuracy by a neural network that takes the form $L_1 \sigma_1 U_1 \sigma_2 L_2 \sigma_3 U_2 \cdots L_r \sigma_{2r-1} U_r$, i.e., where the weight matrices alternate between lower and upper triangular matrices, $\sigma_i(x) := \sigma(x - b_i)$ for some bias vector $b_i$, and the activation $\sigma$ may be chosen to be essentially any uniformly continuous nonpolynomial function. The same result also holds with Toeplitz matrices, i.e., $f \approx T_1 \sigma_1 T_2 \sigma_2 \cdots \sigma_{r-1} T_r$ to arbitrary accuracy, and likewise for Hankel matrices. A consequence of our Toeplitz result is a fixed-width universal approximation theorem for convolutional neural networks, which so far have only arbitrary width versions. Since our results apply in particular to the case when $f$ is a general neural network, we may regard them as LU and Toeplitz decompositions of a neural network. The practical implication of our results is that one may vastly reduce the number of weight parameters in a neural network without sacrificing its power of universal approximation. We will present several experiments on real data sets to show that imposing such structures on the weight matrices sharply reduces the number of training parameters with almost no noticeable effect on test accuracy.
NeuralUDF: Learning Unsigned Distance Fields for Multi-View Reconstruction of Surfaces with Arbitrary Topologies

We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. Recent advances in neural rendering based reconstruction have achieved compelling results. However, these methods are limited to objects with closed surfaces since they adopt Signed Distance Function (SDF) as surface representation which requires the target shape to be divided into inside and outside. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation. Specifically, a new density function that correlates the property of UDF with the volume rendering scheme is introduced for robust optimization of the UDF fields. Experiments on the DTU and DeepFashion3D datasets show that our method not only enables high-quality reconstruction of non-closed shapes with complex typologies, but also achieves comparable performance to the SDF based methods on the reconstruction of closed surfaces. Visit our project page at https://www.xxlong. site/NeuralUDF.
Multiagent MST Cover: Pleasing All Optimally via A Simple Voting Rule

Given a connected graph on whose edges we can build roads to connect the nodes, a number of agents hold possibly different perspectives on which edges should be selected by assigning different edge weights. Our task is to build a minimum number of roads so that every agent has a spanning tree in the built subgraph whose weight is the same as a minimum spanning tree in the original graph. We first show that this problem is NP-hard and does not admit better than ((1-o(1)) ln k)-approximation polynomial-time algorithms unless P = NP, where k is the number of agents. We then give a simple voting algorithm with an optimal approximation ratio. Moreover, our algorithm only needs to access the agents' rankings on the edges. Finally, we extend our problem to submodular objective functions and Matroid rank constraints.
An Hα Impression of Lyα Galaxies at z ≃ 6 with Deep JWST/NIRCam Imaging

We present a study of seven spectroscopically confirmed (Lyα-emitting) galaxies at redshift z ≃ 6 using the JWST/NIRCam imaging data. These galaxies, with a wide range of Lyα luminosities, were recently observed in a series of NIRCam broad and medium bands. We constrain the rest-frame UV/optical continua and measure the Hα line emission of the galaxies using the combination of JWST/NIRCam and archival HST/WFC3 infrared photometry. We further estimate the escape fractions of their Lyα photons ( fescLyα ) and the production efficiency of ionizing photons (ξ ion). Among the sample, six out of seven galaxies have Lyα escape fractions of ≲10%, which might be the status for most of the star-forming galaxies at z ≃ 6. One UV-faint Lyα galaxy with an extremely blue UV slope has a large value of fescLyα reaching ≃50%. These galaxies have a broad range of ξ ion over log10 ξ ion,0 (Hz erg−1) ∼ 25.0–26.5. We find that UV-fainter galaxies with bluer UV-continuum slopes likely have higher escape fractions of Lyα photons. We also find that galaxies with higher Lyα line emission tend to produce ionizing photons more efficiently. The most Lyα-luminous galaxy in the sample has a very high ξ ion,0 of log10 ξ ion,0 (Hz erg−1) > 26. Our results support the scenario that Lyα galaxies may have served as an important contributor to cosmic reionization. Blue and bright Lyα galaxies are excellent targets for JWST follow-up spectroscopic observations.
A Report on the Euphemisms Detection Shared Task

This paper presents The Shared Task on Euphemism Detection for the Third Workshop on Figurative Language Processing (FigLang 2022) held in conjunction with EMNLP 2022. Participants were invited to investigate the euphemism detection task: given input text, identify whether it contains a euphemism. The input data is a corpus of sentences containing potentially euphemistic terms (PETs) collected from the GloWbE corpus, and are human-annotated as containing either a euphemistic or literal usage of a PET. In this paper, we present the results and analyze the common themes, methods and findings of the participating teams.
Graph Contrastive Learning for Materials

Recent work has shown the potential of graph neural networks to efficiently predict material properties, enabling high-throughput screening of materials. Training these models, however, often requires large quantities of labelled data, obtained via costly methods such as ab initio calculations or experimental evaluation. By leveraging a series of material-specific transformations, we introduce CrystalCLR, a framework for constrastive learning of representations with crystal graph neural networks. With the addition of a novel loss function, our framework is able to learn representations competitive with engineered fingerprinting methods. We also demonstrate that via model finetuning, contrastive pretraining can improve the performance of graph neural networks for prediction of material properties and significantly outperform traditional ML models that use engineered fingerprints. Lastly, we observe that CrystalCLR produces material representations that form clusters by compound class.
A motivic integral $p$-adic cohomology

We construct an integral $p$-adic cohomology that compares with rigid cohomology after inverting $p$. Our approach is based on the log-Witt differentials of Hyodo-Kato and log-\'etale motives of Binda-Park-{\O}stv{\ae}r. In case $k$ satisfies resolutions of singularities, we moreover prove that it agrees with the"good"integral $p$-adic cohomology of Ertl-Shiho-Sprang: from this we deduce some interesting motivic properties and a K\"unneth formula for the $p$-adic cohomology of Ertl-Shiho-Sprang.
A Self-Attention Ansatz for Ab-initio Quantum Chemistry

We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schr\"odinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.
Magnetic bead counter using a micro-Hall sensor for biological applications

Micro-Hall sensors have been fabricated, and various numbers of micron-size magnetic beads have been placed within the sensor area. The Hall resistances measured at room temperature are found to be proportional to the number of the beads, and are in good agreement with the numerically simulated results presented in this study. Our sensors are designed to measure the number of beads between zero and full-scale signals for a given number range of interest. The effects of miniaturizing the beads and sensors to nanoscale are also discussed.
Delving into Out-of-Distribution Detection with Vision-Language Representations

Recognizing out-of-distribution (OOD) samples is critical for machine learning systems deployed in the open world. The vast majority of OOD detection methods are driven by a single modality (e.g., either vision or language), leaving the rich information in multi-modal representations untapped. Inspired by the recent success of vision-language pre-training, this paper enriches the landscape of OOD detection from a single-modal to a multi-modal regime. Particularly, we propose Maximum Concept Matching (MCM), a simple yet effective zero-shot OOD detection method based on aligning visual features with textual concepts. We contribute in-depth analysis and theoretical insights to understand the effectiveness of MCM. Extensive experiments demonstrate that MCM achieves superior performance on a wide variety of real-world tasks. MCM with vision-language features outperforms a common baseline with pure visual features on a hard OOD task with semantically similar classes by 13.1% (AUROC). Code is available at https://github.com/deeplearning-wisc/MCM.
Models of Opinion Dynamics with Random Parametrisation

We analyse a model of binary opinion dynamics in which iterative discussions take place in local groups of individuals and study the effects of random deviations from the group majority. The probability of a deviation or flip depends on the magnitude of the majority. Depending on the values of the flip parameters which give the probability of a deviation, the model shows a wide variety of behaviour. We are interested in the characteristics of the model when the flip parameters are themselves randomly selected, following some probability distribution. Examples of these characteristics are whether large majorities and ties are attractors or repulsors, or the number of fixed points in the dynamics of the model. Which of the features of the model are likely to appear? Which ones are unlikely because they only present as events of low probability with respect to the distribution of the flip parameters? Answers to such questions allow us to distinguish mathematical properties which are stable under a variety of assumptions on the distribution of the flip parameters from features which are very rare and thus more of theoretical than practical interest. In this article, we present both exact numerical results for specific distributions of the flip parameters and small discussion groups and rigorous results in the form of limit theorems for large discussion groups. Small discussion groups model friend or work groups - people that personally know each other and frequently spend time together. Large groups represent scenarios such as social media or political entities such as cities, states, or countries.
Meta-Learning the Inductive Bias of Simple Neural Circuits

Training data is always finite, making it unclear how to generalise to unseen situations. But, animals do generalise, wielding Occam's razor to select a parsimonious explanation of their observations. How they do this is called their inductive bias, and it is implicitly built into the operation of animals' neural circuits. This relationship between an observed circuit and its inductive bias is a useful explanatory window for neuroscience, allowing design choices to be understood normatively. However, it is generally very difficult to map circuit structure to inductive bias. Here, we present a neural network tool to bridge this gap. The tool meta-learns the inductive bias by learning functions that a neural circuit finds easy to generalise, since easy-to-generalise functions are exactly those the circuit chooses to explain incomplete data. In systems with analytically known inductive bias, i.e. linear and kernel regression, our tool recovers it. Generally, we show it can flexibly extract inductive biases from supervised learners, including spiking neural networks, and show how it could be applied to real animals. Finally, we use our tool to interpret recent connectomic data illustrating its intended use: understanding the role of circuit features through the resulting inductive bias.
Model-Free Reinforcement Learning with the Decision-Estimation Coefficient

We consider the problem of interactive decision making, encompassing structured bandits and reinforcement learning with general function approximation. Recently, Foster et al. (2021) introduced the Decision-Estimation Coefficient, a measure of statistical complexity that lower bounds the optimal regret for interactive decision making, as well as a meta-algorithm, Estimation-to-Decisions, which achieves upper bounds in terms of the same quantity. Estimation-to-Decisions is a reduction, which lifts algorithms for (supervised) online estimation into algorithms for decision making. In this paper, we show that by combining Estimation-to-Decisions with a specialized form of optimistic estimation introduced by Zhang (2022), it is possible to obtain guarantees that improve upon those of Foster et al. (2021) by accommodating more lenient notions of estimation error. We use this approach to derive regret bounds for model-free reinforcement learning with value function approximation, and give structural results showing when it can and cannot help more generally.
A ‘BACKWARD’ FREE-CONVECTIVE BOUNDARY LAYER

In this paper the cooling of a low-heat-resistance sheet that moves downwards is considered. The free-convective velocities are assumed to be much larger than the velocity of the sheet. As a result the motion of the fluid is mainly towards the point where the sheet enters the system and a ‘backward’ boundary layer ensues. It is shown that the equations can be reduced by a similarity transformation. For intermediate values of the Prandtl number the equations are solved numerically. Matched asymptotic expansions are used to treat the cases of extremely small and extremely large Prandtl number. The Nusselt number associated with this process is compared with those due to other modes of heat transfer, such as radiation and forced convection. The algebraic behaviour of the boundary-layer functions is discussed.
Properties of condensed matter from fundamental physical constants

Fundamental physical constants play a profound role in physics. For example, they govern nuclear reactions, formation of stars, nuclear synthesis and stability of biologically vital elements. These are high-energy processes discussed in particle physics, astronomy and cosmology. More recently, it was realised that fundamental physical constants extend their governing reach to low-energy processes and properties operating in condensed matter systems, often in an unexpected way. These properties are those we experience daily and can routinely measure, including viscosity, thermal conductivity, elasticity and sound. Here, we review this work. We start with the lower bound on liquid viscosity, its origin and show how to relate the bound to fundamental physical constants. The lower bound of kinematic viscosity represents the global minimum on the phase diagram. We show how this result answers the long-standing question considered by Purcell and Weisskopf, namely why viscosity never falls below a certain value. An accompanying insight is that water viscosity and water-based life are well attuned to fundamental constants including the Planck constant. We then discuss viscosity minima in liquid He above and below the λ-point. We subsequently consider a very different property, thermal diffusivity, and show that it has the same minimum fixed by fundamental physical constants as viscosity. We also discuss bounds related to elastic properties, elastic moduli and their analogues in low-dimensional systems, and show how these bounds are related to the upper bound for the speed of sound. We conclude with listing ways in which the discussion of fundamental constants and bounds advance physical theories.
Exact analytic solutions to turbulent particle flow equations

A technique for solving a certain class of partial differential equations is used to obtain exact solutions for particle dispersion in an unbounded Gaussian random flow field in which the mean flow is a simple shear and the turbulence is homogeneous and stationary. Results are compared with those of a simple random walk simulation and those obtained from tracking particles in a simple developing shear generated by LES.
JetFlow: Generating Jets with Conditioned and Mass Constrained Normalising Flows

Fast data generation based on Machine Learning has become a major research topic in particle physics. This is mainly because the Monte Carlo simulation approach is computationally challenging for future colliders, which will have a significantly higher luminosity. The generation of collider data is similar to point cloud generation with complex correlations between the points. In this study, the generation of jets with up to 30 constituents with Normalising Flows using Rational Quadratic Spline coupling layers is investigated. Without conditioning on the jet mass, our Normalising Flows are unable to model all correlations in data correctly, which is evident when comparing the invariant jet mass distributions between ground truth and generated data. Using the invariant mass as a condition for the coupling transformation enhances the performance on all tracked metrics. In addition, we demonstrate how to sample the original mass distribution by interpolating the empirical cumulative distribution function. Similarly, the variable number of constituents is taken care of by introducing an additional condition on the number of constituents in the jet. Furthermore, we study the usefulness of including an additional mass constraint in the loss term. On the \texttt{JetNet} dataset, our model shows state-of-the-art performance combined with fast and stable training.
The Second-place Solution for CVPR 2022 SoccerNet Tracking Challenge

This is our second-place solution for CVPR 2022 SoccerNet Tracking Challenge. Our method mainly includes two steps: online short-term tracking using our Cascaded Buffer-IoU (C-BIoU) Tracker, and, offline long-term tracking using appearance feature and hierarchical clustering. At each step, online tracking yielded HOTA scores near 90, and offline tracking further improved HOTA scores to around 93.2.
Robustness Analysis of Deep Learning Models for Population Synthesis

Deep generative models have become useful for synthetic data generation, particularly population synthesis. The models implicitly learn the probability distribution of a dataset and can draw samples from a distribution. Several models have been proposed, but their performance is only tested on a single cross-sectional sample. The implementation of population synthesis on single datasets is seen as a drawback that needs further studies to explore the robustness of the models on multiple datasets. While comparing with the real data can increase trust and interpretability of the models, techniques to evaluate deep generative models’ robustness for population synthesis remain underexplored. In this study, we present bootstrap confidence interval for the deep generative models, an approach that computes efficient confidence intervals for mean errors predictions to evaluate the robustness of the models to multiple datasets. Specifically, we adopt the tabular-based Composite Travel Generative Adversarial Network (CTGAN) and Variational Autoencoder (VAE), to estimate the distribution of the population, by generating agents that have tabular data using several samples over time from the same study area. The models are implemented on multiple travel diaries of Montre´al Origin-Destination Survey of 2008, 2013, and 2018 and compare the predictive performance under varying sample sizes from multiple surveys. Results show that the predictive errors of CTGAN have narrower confidence intervals indicating its robustness to multiple datasets of the varying sample sizes when compared to VAE. Again, the evaluation of model robustness against varying sample size shows a minimal decrease in model performance with decrease in sample size. This study directly supports agent-based modelling by enabling finer synthetic generation of populations in a reliable environment. analyse the robustness of deep generative models to changes in varying populations. In this study, we adopt the tabular-based Composite Travel GAN (CTGAN) that outputs selected tabular attributes and the Tabular Variational Autoencoder. We compare similarity in statistical information of the synthesized results of the tabular-based CTGAN and the VAE based on activity diaries of three different surveys. Thus, the model is trained on varying sample population from Montreal origin-destination survey in 2008, 2013, and 2018. The CTGAN outperforms the VAE in all the observations with minimum error rates. We evaluate the robustness of the deep generative models based on variation in sample population. Even though an incremental change in sample population slightly decreases the error rate, it cannot be the basis for performance of the model. Subsequently, we use bootstrapping method to estimate the confidence intervals of the models by determining the mean of the error metrics from more populations. We demonstrate that our model can generate synthesized data on different datasets at varying sample sizes while maintaining minimal error.
On the energy scaling behaviour of singular perturbation models with prescribed Dirichlet data involving higher order laminates

Motivated by complex microstructures in the modelling of shape-memory alloys and by rigidity and flexibility considerations for the associated differential inclusions, in this article we study the energy scaling behaviour of a simplified $m$-well problem without gauge invariances. Considering wells for which the lamination convex hull consists of one-dimensional line segments of increasing order of lamination, we prove that for prescribed Dirichlet data the energy scaling is determined by the \emph{order of lamination of the Dirichlet data}. This follows by deducing (essentially) matching upper and lower scaling bounds. For the \emph{upper} bound we argue by providing iterated branching constructions, and complement this with ansatz-free \emph{lower} bounds. These are deduced by a careful analysis of the Fourier multipliers of the associated energies and iterated ``bootstrap arguments'' based on the ideas from \cite{RT21}. Relying on these observations, we study models involving laminates of arbitrary order.