Miracle-dz/UniFAR-FTUs huggingface dataset:
Statistical Inference on Simulation Output: Batching as an Inferential Device

We present {batching} as an omnibus device for statistical inference on simulation output. We consider the classical context of a simulationist performing statistical inference on an estimator $\theta_n$ (of an unknown fixed quantity $\theta$) using only the output data $(Y_1,Y_2,\ldots,Y_n)$ gathered from a simulation. By \emph{statistical inference}, we mean approximating the sampling distribution of the error $\theta_n-\theta$ toward: (A) estimating an ``assessment'' functional $\psi$, e.g., bias, variance, or quantile; or (B) constructing a $(1-\alpha)$-confidence region on $\theta$. We argue that batching is a remarkably simple and effective inference device that is especially suited for handling dependent output data such as what one frequently encounters in simulation contexts. We demonstrate that if the number of batches and the extent of their overlap are chosen correctly, batching retains bootstrap's attractive theoretical properties of {strong consistency} and {higher-order accuracy}. For constructing confidence regions, we characterize two limiting distributions associated with a Studentized statistic. Our extensive numerical experience confirms theoretical insight, especially about the effects of batch size and batch overlap.
Deep Hashing via Householder Quantization

Hashing is at the heart of large-scale image similarity search, and recent methods have been substantially improved through deep learning techniques. Such algorithms typically learn continuous embeddings of the data. To avoid a subsequent costly binarization step, a common solution is to employ loss functions that combine a similarity learning term (to ensure similar images are grouped to nearby embeddings) and a quantization penalty term (to ensure that the embedding entries are close to binarized entries, e.g., -1 or 1). Still, the interaction between these two terms can make learning harder and the embeddings worse. We propose an alternative quantization strategy that decomposes the learning problem in two stages: first, perform similarity learning over the embedding space with no quantization; second, find an optimal orthogonal transformation of the embeddings so each coordinate of the embedding is close to its sign, and then quantize the transformed embedding through the sign function. In the second step, we parametrize orthogonal transformations using Householder matrices to efficiently leverage stochastic gradient descent. Since similarity measures are usually invariant under orthogonal transformations, this quantization strategy comes at no cost in terms of performance. The resulting algorithm is unsupervised, fast, hyperparameter-free and can be run on top of any existing deep hashing or metric learning algorithm. We provide extensive experimental results showing that this approach leads to state-of-the-art performance on widely used image datasets, and, unlike other quantization strategies, brings consistent improvements in performance to existing deep hashing algorithms.
Positive Competitive Networks for Sparse Reconstruction

We propose and analyze a continuous-time firing-rate neural network, the positive firing-rate competitive network (\pfcn), to tackle sparse reconstruction problems with non-negativity constraints. These problems, which involve approximating a given input stimulus from a dictionary using a set of sparse (active) neurons, play a key role in a wide range of domains, including for example neuroscience, signal processing, and machine learning. First, by leveraging the theory of proximal operators, we relate the equilibria of a family of continuous-time firing-rate neural networks to the optimal solutions of sparse reconstruction problems. Then, we prove that the \pfcn is a positive system and give rigorous conditions for the convergence to the equilibrium. Specifically, we show that the convergence: (i) only depends on a property of the dictionary; (ii) is linear-exponential, in the sense that initially the convergence rate is at worst linear and then, after a transient, it becomes exponential. We also prove a number of technical results to assess the contractivity properties of the neural dynamics of interest. Our analysis leverages contraction theory to characterize the behavior of a family of firing-rate competitive networks for sparse reconstruction with and without non-negativity constraints. Finally, we validate the effectiveness of our approach via a numerical example.
Context Shift Reduction for Offline Meta-Reinforcement Learning

Offline meta-reinforcement learning (OMRL) utilizes pre-collected offline datasets to enhance the agent's generalization ability on unseen tasks. However, the context shift problem arises due to the distribution discrepancy between the contexts used for training (from the behavior policy) and testing (from the exploration policy). The context shift problem leads to incorrect task inference and further deteriorates the generalization ability of the meta-policy. Existing OMRL methods either overlook this problem or attempt to mitigate it with additional information. In this paper, we propose a novel approach called Context Shift Reduction for OMRL (CSRO) to address the context shift problem with only offline datasets. The key insight of CSRO is to minimize the influence of policy in context during both the meta-training and meta-test phases. During meta-training, we design a max-min mutual information representation learning mechanism to diminish the impact of the behavior policy on task representation. In the meta-test phase, we introduce the non-prior context collection strategy to reduce the effect of the exploration policy. Experimental results demonstrate that CSRO significantly reduces the context shift and improves the generalization ability, surpassing previous methods across various challenging domains.
Exploitation-Guided Exploration for Semantic Embodied Navigation

In the recent progress in embodied navigation and sim-to-robot transfer, modular policies have emerged as a de facto framework. However, there is more to compositionality beyond the decomposition of the learning load into modular components. In this work, we investigate a principled way to syntactically combine these components. Particularly, we propose Exploitation-Guided Exploration (XGX) where separate modules for exploration and exploitation come together in a novel and intuitive manner. We configure the exploitation module to take over in the deterministic final steps of navigation i.e. when the goal becomes visible. Crucially, an exploitation module teacher-forces the exploration module and continues driving an overridden policy optimization. XGX, with effective decomposition and novel guidance, improves the state-of-the-art performance on the challenging object navigation task from 70% to 73%. Along with better accuracy, through targeted analysis, we show that XGX is also more efficient at goal-conditioned exploration. Finally, we show sim-to-real transfer to robot hardware and XGX performs over two-fold better than the best baseline from simulation benchmarking. Project page: xgxvisnav.github. io
Personality Style Recognition via Machine Learning: Identifying Anaclitic and Introjective Personality Styles from Patients' Speech

In disentangling the heterogeneity observed in psychopathology, personality of the patients is considered crucial. While it has been demonstrated that personality traits are reflected in the language used by a patient, we hypothesize that this enables automatic inference of the personality type directly from speech utterances, potentially more accurately than through a traditional questionnaire-based approach explicitly designed for personality classification. To validate this hypothesis, we adopt natural language processing (NLP) and standard machine learning tools for classification. We test this on a dataset of recorded clinical diagnostic interviews (CDI) on a sample of 79 patients diagnosed with major depressive disorder (MDD) -- a condition for which differentiated treatment based on personality styles has been advocated -- and classified into anaclitic and introjective personality styles. We start by analyzing the interviews to see which linguistic features are associated with each style, in order to gain a better understanding of the styles. Then, we develop automatic classifiers based on (a) standardized questionnaire responses; (b) basic text features, i.e., TF-IDF scores of words and word sequences; (c) more advanced text features, using LIWC (linguistic inquiry and word count) and context-aware features using BERT (bidirectional encoder representations from transformers); (d) audio features. We find that automated classification with language-derived features (i.e., based on LIWC) significantly outperforms questionnaire-based classification models. Furthermore, the best performance is achieved by combining LIWC with the questionnaire features. This suggests that more work should be put into developing linguistically based automated techniques for characterizing personality, however questionnaires still to some extent complement such methods.
On polynomial degree-boundedness

We prove a conjecture of Bonamy, Bousquet, Pilipczuk, Rz\k{a}\. zewski, Thomass\'e, and Walczak, that for every graph $H$, there is a polynomial $p$ such that for every positive integer $s$, every graph of average degree at least $p(s)$ contains either $K_{s,s}$ as a subgraph or contains an induced subdivision of $H$. This improves upon a result of K\"uhn and Osthus from 2004 who proved it for graphs whose average degree is at least triply exponential in $s$ and a recent result of Du, Gir\~{a}o, Hunter, McCarty and Scott for graphs with average degree at least singly exponential in $s$. As an application, we prove that the class of graphs that do not contain an induced subdivision of $K_{s,t}$ is polynomially $\chi$-bounded. In the case of $K_{2,3}$, this is the class of theta-free graphs, and answers a question of Davies. Along the way, we also answer a recent question of McCarty, by showing that if $\mathcal{G}$ is a hereditary class of graphs for which there is a polynomial $p$ such that every bipartite $K_{s,s}$-free graph in $\mathcal{G}$ has average degree at most $p(s)$, then more generally, there is a polynomial $p'$ such that every $K_{s,s}$-free graph in $\mathcal{G}$ has average degree at most $p'(s)$. Our main new tool is an induced variant of the K\H{o}v\'ari-S\'os-Tur\'an theorem, which we find to be of independent interest.
Parikh’s Theorem Made Symbolic

Parikh’s Theorem is a fundamental result in automata theory with numerous applications in computer science. These include software verification (e.g. infinite-state verification, string constraints, and theory of arrays), verification of cryptographic protocols (e.g. using Horn clauses modulo equational theories) and database querying (e.g. evaluating path-queries in graph databases), among others. Parikh’s Theorem states that the letter-counting abstraction of a language recognized by finite automata or context-free grammars is definable in Linear Integer Arithmetic (a.k.a. Presburger Arithmetic). In fact, there is a linear-time algorithm computing existential Presburger formulas capturing such abstractions, which enables an efficient analysis via SMT-solvers. Unfortunately, real-world applications typically require large alphabets (e.g. Unicode, containing a million of characters) — which are well-known to be not amenable to explicit treatment of the alphabets — or even worse infinite alphabets. Symbolic automata have proven in the last decade to be an effective algorithmic framework for handling large finite or even infinite alphabets. A symbolic automaton employs an effective boolean algebra, which offers a symbolic representation of character sets (i.e. in terms of predicates) and often lends itself to an exponentially more succinct representation of a language. Instead of letter-counting, Parikh’s Theorem for symbolic automata amounts to counting the number of times different predicates are satisfied by an input sequence. Unfortunately, naively applying Parikh’s Theorem from classical automata theory to symbolic automata yields existential Presburger formulas of exponential size. In this paper, we provide a new construction for Parikh’s Theorem for symbolic automata and grammars, which avoids this exponential blowup: our algorithm computes an existential formula in polynomial-time over (quantifier-free) Presburger and the base theory. In fact, our algorithm extends to the model of parametric symbolic grammars, which are one of the most expressive models of languages over infinite alphabets. We have implemented our algorithm and show it can be used to solve string constraints that are difficult to solve by existing solvers.
Shortcomings of using the SCAN functional for deep defects and small polarons in semiconductors

We find the recently developed strongly constrained and appropriately normalized (SCAN) functional, now widely used in calculations of many materials, is not able to reliably describe the properties of deep defects and small polarons in several semiconductors and insulators. These shortcomings are due to a failure to identify the large structural distortions and charge localization that accompanies the presence of defects and the formation of small polarons.
On an Optimal Stopping Problem with a Discontinuous Reward

We study an optimal stopping problem with an unbounded, time-dependent and discontinuous reward function.This problem is motivated by the pricing of a variable annuity contract with guaranteed minimum maturity benefit, under the assumption that the policyholder's surrender behaviour maximizes the risk-neutral value of the contract. We consider a general fee and surrender charge function, and give a condition under which optimal stopping always occurs at maturity. Using an alternative representation for the value function of the optimization problem, we study its analytical properties and the resulting surrender (or exercise) region. In particular, we show that the non-emptiness and the shape of the surrender region are fully characterized by the fee and the surrender charge functions, which provides a powerful tool to understand their interrelation and how it affects early surrenders and the optimal surrender boundary. Under certain conditions on these two functions, we develop three representations for the value function; two are analogous to their American option counterpart, and one is new to the actuarial and American option pricing literature.
Random Field Augmentations for Self-Supervised Representation Learning

Self-supervised representation learning is heavily dependent on data augmentations to specify the invariances encoded in representations. Previous work has shown that applying diverse data augmentations is crucial to downstream performance, but augmentation techniques remain under-explored. In this work, we propose a new family of local transformations based on Gaussian random fields to generate image augmentations for self-supervised representation learning. These transformations generalize the well-established affine and color transformations (translation, rotation, color jitter, etc.) and greatly increase the space of augmentations by allowing transformation parameter values to vary from pixel to pixel. The parameters are treated as continuous functions of spatial coordinates, and modeled as independent Gaussian random fields. Empirical results show the effectiveness of the new transformations for self-supervised representation learning. Specifically, we achieve a 1.7% top-1 accuracy improvement over baseline on ImageNet downstream classification, and a 3.6% improvement on out-of-distribution iNaturalist downstream classification. However, due to the flexibility of the new transformations, learned representations are sensitive to hyperparameters. While mild transformations improve representations, we observe that strong transformations can degrade the structure of an image, indicating that balancing the diversity and strength of augmentations is important for improving generalization of learned representations.
Microstructure effects on the thermal fatigue of solder joints: coupling damage measurements with multi-scale modelling

Thermal fatigue is a common failure mode in electronic solder joints, yet the role of microstructure is incompletely understood. Here, we quantify the evolution of microstructure and damage in Sn-3Ag-0.5Cu joints throughout a ball grid array (BGA) package using electron backscatter diffraction (EBSD) mapping of localised subgrains, recrystallisation and heavily coarsened Ag3Sn. We then interpret the results with a multi-scale modelling approach that links from a continuum model at the package/board scale through to a crystal plasticity finite element model at the microstructure scale accounting for the anisotropic thermal expansion, elastic and plastic properties of beta-Sn with embedded Ag3Sn and Cu6Sn5 particles. We measure and explain the dependence of damage evolution on (i) the beta-Sn crystal orientation(s) in single and multigrain joints, and (ii) the coefficient of thermal expansion (CTE) mismatch between tin grains in cyclic twinned multigrain joints. We further explore the relative importance of the solder microstructure versus the joint location in the array. The results provide a basis for designing optimum solder joint microstructures for thermal fatigue resistance.
Meta-Adapter: An Online Few-shot Learner for Vision-Language Model

The contrastive vision-language pre-training, known as CLIP, demonstrates remarkable potential in perceiving open-world visual concepts, enabling effective zero-shot image recognition. Nevertheless, few-shot learning methods based on CLIP typically require offline fine-tuning of the parameters on few-shot samples, resulting in longer inference time and the risk of over-fitting in certain domains. To tackle these challenges, we propose the Meta-Adapter, a lightweight residual-style adapter, to refine the CLIP features guided by the few-shot samples in an online manner. With a few training samples, our method can enable effective few-shot learning capabilities and generalize to unseen data or tasks without additional fine-tuning, achieving competitive performance and high efficiency. Without bells and whistles, our approach outperforms the state-of-the-art online few-shot learning method by an average of 3.6\% on eight image classification datasets with higher inference speed. Furthermore, our model is simple and flexible, serving as a plug-and-play module directly applicable to downstream tasks. Without further fine-tuning, Meta-Adapter obtains notable performance improvements in open-vocabulary object detection and segmentation tasks.
TS-Diffusion: Generating Highly Complex Time Series with Diffusion Models

While current generative models have achieved promising performances in time-series synthesis, they either make strong assumptions on the data format (e.g., regularities) or rely on pre-processing approaches (e.g., interpolations) to simplify the raw data. In this work, we consider a class of time series with three common bad properties, including sampling irregularities, missingness, and large feature-temporal dimensions, and introduce a general model, TS-Diffusion, to process such complex time series. Our model consists of three parts under the framework of point process. The first part is an encoder of the neural ordinary differential equation (ODE) that converts time series into dense representations, with the jump technique to capture sampling irregularities and self-attention mechanism to handle missing values; The second component of TS-Diffusion is a diffusion model that learns from the representation of time series. These time-series representations can have a complex distribution because of their high dimensions; The third part is a decoder of another ODE that generates time series with irregularities and missing values given their representations. We have conducted extensive experiments on multiple time-series datasets, demonstrating that TS-Diffusion achieves excellent results on both conventional and complex time series and significantly outperforms previous baselines.
Strong constraints on the gravitational law from $Gaia$ DR3 wide binaries

We test Milgromian dynamics (MOND) using wide binary stars (WBs) with separations of $2-30$ kAU. Locally, the WB orbital velocity in MOND should exceed the Newtonian prediction by $\approx 20\%$ at asymptotically large separations given the Galactic external field effect (EFE). We investigate this with a detailed statistical analysis of $Gaia$ DR3 data on 8611 WBs within 250 pc of the Sun. Orbits are integrated in a rigorously calculated gravitational field that directly includes the EFE. We also allow line of sight contamination and undetected close binary companions to the stars in each WB. We interpolate between the Newtonian and Milgromian predictions using the parameter $\alpha_{\rm{grav}}$, with 0 indicating Newtonian gravity and 1 indicating MOND. Directly comparing the best Newtonian and Milgromian models reveals that Newtonian dynamics is preferred at $19\sigma$ confidence. Using a complementary Markov Chain Monte Carlo analysis, we find that $\alpha_{\rm{grav}} = -0.021^{+0.065}_{-0.045}$, which is fully consistent with Newtonian gravity but excludes MOND at $16\sigma$ confidence. This is in line with the similar result of Pittordis and Sutherland using a somewhat different sample selection and less thoroughly explored population model. We show that although our best-fitting model does not fully reproduce the observations, an overwhelmingly strong preference for Newtonian gravity remains in a considerable range of variations to our analysis. Adapting the MOND interpolating function to explain this result would cause tension with rotation curve constraints. We discuss the broader implications of our results in light of other works, concluding that MOND must be substantially modified on small scales to account for local WBs.
Do LLMs exhibit human-like response biases? A case study in survey design

As large language models (LLMs) become more capable, there is growing excitement about the possibility of using LLMs as proxies for humans in real-world tasks where subjective labels are desired, such as in surveys and opinion polling. One widely-cited barrier to the adoption of LLMs as proxies for humans in subjective tasks is their sensitivity to prompt wording - but interestingly, humans also display sensitivities to instruction changes in the form of response biases. We investigate the extent to which LLMs reflect human response biases, if at all. We look to survey design, where human response biases caused by changes in the wordings of"prompts"have been extensively explored in social psychology literature. Drawing from these works, we design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior, particularly in models that have undergone RLHF. Furthermore, even if a model shows a significant change in the same direction as humans, we find that they are sensitive to perturbations that do not elicit significant changes in humans. These results highlight the pitfalls of using LLMs as human proxies, and underscore the need for finer-grained characterizations of model behavior. Our code, dataset, and collected samples are available at https://github.com/lindiatjuatja/BiasMonkey