Miracle-dz/UniFAR-FTUs huggingface dataset:
The Interpolating Information Criterion for Overparameterized Models

The problem of model selection is considered for the setting of interpolating estimators, where the number of model parameters exceeds the size of the dataset. Classical information criteria typically consider the large-data limit, penalizing model size. However, these criteria are not appropriate in modern settings where overparameterized models tend to perform well. For any overparameterized model, we show that there exists a dual underparameterized model that possesses the same marginal likelihood, thus establishing a form of Bayesian duality. This enables more classical methods to be used in the overparameterized setting, revealing the Interpolating Information Criterion, a measure of model quality that naturally incorporates the choice of prior into the model selection. Our new information criterion accounts for prior misspecification, geometric and spectral properties of the model, and is numerically consistent with known empirical and theoretical behavior in this regime.
$ \mathrm{SE} (3) $ Synchronization by Eigenvectors of Dual Quaternion Matrices

In synchronization problems, the goal is to estimate elements of a group from noisy measurements of their ratios. A popular estimation method for synchronization is the spectral method. It extracts the group elements from eigenvectors of a block matrix formed from the measurements. The eigenvectors must be projected, or"rounded", onto the group. The rounding procedures are constructed ad hoc and increasingly so when applied to synchronization problems over non-compact groups. In this paper, we develop a spectral approach to synchronization over the non-compact group $\mathrm{SE}(3)$, the group of rigid motions of $\mathbb{R}^3$. We based our method on embedding $\mathrm{SE}(3)$ into the algebra of dual quaternions, which has deep algebraic connections with the group $\mathrm{SE}(3)$. These connections suggest a natural rounding procedure considerably more straightforward than the current state-of-the-art for spectral $\mathrm{SE}(3)$ synchronization, which uses a matrix embedding of $\mathrm{SE}(3)$. We show by numerical experiments that our approach yields comparable results to the current state-of-the-art in $\mathrm{SE}(3)$ synchronization via the spectral method. Thus, our approach reaps the benefits of the dual quaternion embedding of $\mathrm{SE}(3)$, while yielding estimators of similar quality.
Riesz feature representation: scale equivariant scattering network for classification tasks

Scattering networks yield powerful and robust hierarchical image descriptors which do not require lengthy training and which work well with very few training data. However, they rely on sampling the scale dimension. Hence, they become sensitive to scale variations and are unable to generalize to unseen scales. In this work, we define an alternative feature representation based on the Riesz transform. We detail and analyze the mathematical foundations behind this representation. In particular, it inherits scale equivariance from the Riesz transform and completely avoids sampling of the scale dimension. Additionally, the number of features in the representation is reduced by a factor four compared to scattering networks. Nevertheless, our representation performs comparably well for texture classification with an interesting addition: scale equivariance. Our method yields superior performance when dealing with scales outside of those covered by the training dataset. The usefulness of the equivariance property is demonstrated on the digit classification task, where accuracy remains stable even for scales four times larger than the one chosen for training. As a second example, we consider classification of textures.
Torus quotients of Richardson varieties

ABSTRACT In this paper, for any simple, simply connected algebraic group G of type A and for any maximal parabolic subgroup P of G, we provide a criterion for a Richardson variety in G∕P to admit semistable points for the action of a maximal torus T with respect to an ample line bundle on G∕P. We study the structure of the Geometric invariant theory (GIT) quotient for the action of a maximal torus on the Richardson varieties in G2,n with respect to a suitable line bundle on G2,n. We also give a criterion for the Richardson varieties in to admit a semistable point with respect to the line bundle corresponding to the highest root.
Preconditioning techniques for generalized Sylvester matrix equations

Sylvester matrix equations are ubiquitous in scientific computing. However, few solution techniques exist for their generalized multiterm version, as they now arise in an increasingly large number of applications. In this work, we consider algebraic parameter-free preconditioning techniques for the iterative solution of generalized multiterm Sylvester equations. They consist in constructing low Kronecker rank approximations of either the operator itself or its inverse. While the former requires solving standard Sylvester equations in each iteration, the latter only requires matrix-matrix multiplications, which are highly optimized on modern computer architectures. Moreover, low Kronecker rank approximate inverses can be easily combined with sparse approximate inverse techniques, thereby enhancing their performance with little or no damage to their effectiveness.
HYTREL: Hypergraph-enhanced Tabular Data Representation Learning

Language models pretrained on large collections of tabular data have demonstrated their effectiveness in several downstream tasks. However, many of these models do not take into account the row/column permutation invariances, hierarchical structure, etc. that exist in tabular data. To alleviate these limitations, we propose HYTREL, a tabular language model, that captures the permutation invariances and three more structural properties of tabular data by using hypergraphs - where the table cells make up the nodes and the cells occurring jointly together in each row, column, and the entire table are used to form three different types of hyperedges. We show that HYTREL is maximally invariant under certain conditions for tabular data, i.e., two tables obtain the same representations via HYTREL iff the two tables are identical up to permutations. Our empirical results demonstrate that HYTREL consistently outperforms other competitive baselines on four downstream tasks with minimal pretraining, illustrating the advantages of incorporating the inductive biases associated with tabular data into the representations. Finally, our qualitative analyses showcase that HYTREL can assimilate the table structures to generate robust representations for the cells, rows, columns, and the entire table.
Neural Image Compression: Generalization, Robustness, and Spectral Biases

Recent advances in neural image compression (NIC) have produced models that are starting to outperform classic codecs. While this has led to growing excitement about using NIC in real-world applications, the successful adoption of any machine learning system in the wild requires it to generalize (and be robust) to unseen distribution shifts at deployment. Unfortunately, current research lacks comprehensive datasets and informative tools to evaluate and understand NIC performance in real-world settings. To bridge this crucial gap, first, this paper presents a comprehensive benchmark suite to evaluate the out-of-distribution (OOD) performance of image compression methods. Specifically, we provide CLIC-C and Kodak-C by introducing 15 corruptions to the popular CLIC and Kodak benchmarks. Next, we propose spectrally-inspired inspection tools to gain deeper insight into errors introduced by image compression methods as well as their OOD performance. We then carry out a detailed performance comparison of several classic codecs and NIC variants, revealing intriguing findings that challenge our current understanding of the strengths and limitations of NIC. Finally, we corroborate our empirical findings with theoretical analysis, providing an in-depth view of the OOD performance of NIC and its dependence on the spectral properties of the data. Our benchmarks, spectral inspection tools, and findings provide a crucial bridge to the real-world adoption of NIC. We hope that our work will propel future efforts in designing robust and generalizable NIC methods. Code and data will be made available at https://github.com/klieberman/ood_nic.
The Resume Paradox: Greater Language Differences, Smaller Pay Gaps

Over the past decade, the gender pay gap has remained steady with women earning 84 cents for every dollar earned by men on average. Many studies explain this gap through demand-side bias in the labor market represented through employers' job postings. However, few studies analyze potential bias from the worker supply-side. Here, we analyze the language in millions of US workers' resumes to investigate how differences in workers' self-representation by gender compare to differences in earnings. Across US occupations, language differences between male and female resumes correspond to 11% of the variation in gender pay gap. This suggests that females' resumes that are semantically similar to males' resumes may have greater wage parity. However, surprisingly, occupations with greater language differences between male and female resumes have lower gender pay gaps. A doubling of the language difference between female and male resumes results in an annual wage increase of $2,797 for the average female worker. This result holds with controls for gender-biases of resume text and we find that per-word bias poorly describes the variance in wage gap. The results demonstrate that textual data and self-representation are valuable factors for improving worker representations and understanding employment inequities.
Impact of committed minorities: Unveiling critical mass of cooperation in the iterated prisoner's dilemma game

The critical mass effect is a prevailing topic in the study of complex systems. Recent research has shown that a minority of zealots can effectively drive widespread cooperation in social dilemma games. However, achieving a critical mass of cooperation in the prisoner's dilemma requires stricter conditions. The underlying mechanism behind this effect remains unclear, particularly in the context of repeated interactions. This paper aims to investigate the influence of a committed minority on cooperation in the Iterated Prisoner's Dilemma game, a widely studied model of repeated interactions between individuals facing a social dilemma. In contrast to previous findings, we identify tipping points for both well-mixed and structured populations. Our findings demonstrate that a committed minority of unconditional cooperators can induce full cooperation under weak imitation conditions. Conversely, a committed minority of conditional cooperators, who often employ Tit-for-Tat or extortion strategies, can promote widespread cooperation under strong imitation conditions. These results hold true across various network topologies and imitation rules, suggesting that critical mass effects may be a universal principle in social dilemma games. Additionally, we discover that an excessive density of committed conditional cooperators can hinder cooperation in structured populations. This research advances our understanding of the role of committed minorities in shaping social behavior and provides valuable insights into cooperation dynamics.
Heavy quark $\kappa$ and jet $\hat{q}$ transport coefficients in the Glasma early stage of heavy-ion collisions

We study the impact of the Glasma fields, used to describe the very early stage of heavy-ion collisions, on the transport of hard probes, namely heavy quarks and jets. We perform numerical simulations of the strong classical fields using techniques from real-time lattice gauge theory. The resulting fields are used as background for the classical transport of ensembles of particles, described by Wong's equations. For this purpose, we develop a numerical solver for the transport of the probes, based on colored particle-in-cell methods. We focus on the dynamics of heavy quarks and jets in the classical colored fields. To quantify the effect of the Glasma, we extract the momentum broadening of hard probes and evaluate the anisotropy transfer from the Glasma to the probes. Lastly, we evaluate the heavy quark $\kappa$ and jet $\hat{q}$ transport coefficients in the Glasma, which turn out to be large and exhibit a peak, irrespective of the particle initialization.
Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlation

Robustness has been extensively studied in reinforcement learning (RL) to handle various forms of uncertainty such as random perturbations, rare events, and malicious attacks. In this work, we consider one critical type of robustness against spurious correlation, where different portions of the state do not have correlations induced by unobserved confounders. These spurious correlations are ubiquitous in real-world tasks, for instance, a self-driving car usually observes heavy traffic in the daytime and light traffic at night due to unobservable human activity. A model that learns such useless or even harmful correlation could catastrophically fail when the confounder in the test case deviates from the training one. Although motivated, enabling robustness against spurious correlation poses significant challenges since the uncertainty set, shaped by the unobserved confounder and causal structure, is difficult to characterize and identify. Existing robust algorithms that assume simple and unstructured uncertainty sets are therefore inadequate to address this challenge. To solve this issue, we propose Robust State-Confounded Markov Decision Processes (RSC-MDPs) and theoretically demonstrate its superiority in avoiding learning spurious correlations compared with other robust RL counterparts. We also design an empirical algorithm to learn the robust optimal policy for RSC-MDPs, which outperforms all baselines in eight realistic self-driving and manipulation tasks.
NaMemo2: Facilitating Teacher-Student Interaction with Theory-Based Design and Student Autonomy Consideration

Teacher-student interaction (TSI) is essential for learning efficiency and harmonious teacher-student interpersonal relationships. However, studies on TSI support tools often focus on teacher needs while neglecting student needs and autonomy. To enhance both lecturer competence in delivering interpersonal interaction and student autonomy in TSI, we developed NaMemo2, a novel augmented-reality system that allows students to express their willingness to TSI and displays student information to teachers during lectures. The design and evaluation process follows a new framework, STUDIER, which can facilitate the development of theory-based ethnics-aware TSI support tools in general. The quantitative results of our four-week field study with four classes in a university suggested that NaMemo2 can improve 1) TSI in the classroom from both teacher and student perspectives, 2) student attitudes and willingness to TSI, and 3) student attitudes to the deployment of NaMemo2. The qualitative feedback from students and teachers indicated that improving TSI may be responsible for improved attention in students and a better classroom atmosphere during lectures.
Level set methods for gradient-free optimization of metasurface arrays

Global optimization techniques are increasingly preferred over human-driven methods in the design of electromagnetic structures such as metasurfaces, and careful construction and parameterization of the physical structure is critical in ensuring computational efficiency and convergence of the optimization algorithm to a globally optimal solution. While many design variables in physical systems take discrete values, optimization algorithms often benefit from a continuous design space. This work demonstrates the use of level set functions as a continuous basis for designing material distributions for metasurface arrays and introduces an improved parameterization which is termed the periodic level set function. We explore the use of alternate norms in the definition of the level set function and define a new pseudo-inverse technique for upsampling basis coefficients with these norms. The level set method is compared to the fragmented parameterization and shows improved electromagnetic responses for two dissimilar cost functions: a narrowband objective and a broadband objective. Finally, we manufacture an optimized level set metasurface and measure its scattering parameters to demonstrate real-world performance.
Unleashing the Potential of LLMs for Quantum Computing: A Study in Quantum Architecture Design

Large Language Models (LLMs) contribute significantly to the development of conversational AI and has great potentials to assist the scientific research in various areas. This paper attempts to address the following questions: What opportunities do the current generation of generative pre-trained transformers (GPTs) offer for the developments of noisy intermediate-scale quantum (NISQ) technologies? Additionally, what potentials does the forthcoming generation of GPTs possess to push the frontier of research in fault-tolerant quantum computing (FTQC)? In this paper, we implement a QGAS model, which can rapidly propose promising ansatz architectures and evaluate them with application benchmarks including quantum chemistry and quantum finance tasks. Our results demonstrate that after a limited number of prompt guidelines and iterations, we can obtain a high-performance ansatz which is able to produce comparable results that are achieved by state-of-the-art quantum architecture search methods. This study provides a simple overview of GPT's capabilities in supporting quantum computing research while highlighting the limitations of the current GPT at the same time. Additionally, we discuss futuristic applications for LLM in quantum research.
PAT: Parallel Attention Transformer for Visual Question Answering in Vietnamese

We present in this paper a novel scheme for multimodal learning named the Parallel Attention mechanism. In addition, to take into account the advantages of grammar and context in Vietnamese, we propose the Hierarchical Linguistic Features Extractor instead of using an LSTM network to extract linguistic features. Based on these two novel modules, we introduce the Parallel Attention Transformer (PAT), achieving the best accuracy compared to all baselines on the benchmark ViVQA dataset and other SOTA methods including SAAA and MCAN.
Identifying Vulnerable Third-Party Java Libraries from Textual Descriptions of Vulnerabilities and Libraries

To address security vulnerabilities arising from third-party libraries, security researchers maintain databases monitoring and curating vulnerability reports. Application developers can identify vulnerable libraries by directly querying the databases with their used libraries. However, the querying results of vulnerable libraries are not reliable due to the incompleteness of vulnerability reports. Thus, current approaches model the task of identifying vulnerable libraries as a named-entity-recognition (NER) task or an extreme multi-label learning (XML) task. These approaches suffer from highly inaccurate results in identifying vulnerable libraries with complex and similar names, e.g., Java libraries. To address these limitations, in this paper, we propose VulLibMiner, the first to identify vulnerable libraries from textual descriptions of both vulnerabilities and libraries, together with VulLib, a Java vulnerability dataset with their affected libraries. VulLibMiner consists of a TF-IDF matcher to efficiently screen out a small set of candidate libraries and a BERT-FNN model to identify vulnerable libraries from these candidates effectively. We evaluate VulLibMiner using four state-of-the-art/practice approaches of identifying vulnerable libraries on both their dataset named VeraJava and our VulLib dataset. Our evaluation results show that VulLibMiner can effectively identify vulnerable libraries with an average F1 score of 0.657 while the state-of-the-art/practice approaches achieve only 0.521.
Secure Composition of Robust and Optimising Compilers

To ensure that secure applications do not leak their secrets, they are required to uphold several security properties such as spatial and temporal memory safety as well as cryptographic constant time. Existing work shows how to enforce these properties individually, in an architecture-independent way, by using secure compiler passes that each focus on an individual property. Unfortunately, given two secure compiler passes that each preserve a possibly different security property, it is unclear what kind of security property is preserved by the composition of those secure compiler passes. This paper is the first to study what security properties are preserved across the composition of different secure compiler passes. Starting from a general theory of property composition for security-relevant properties (such as the aforementioned ones), this paper formalises a theory of composition of secure compilers. Then, it showcases this theory a secure multi-pass compiler that preserves the aforementioned security-relevant properties. Crucially, this paper derives the security of the multi-pass compiler from the composition of the security properties preserved by its individual passes, which include security-preserving as well as optimisation passes. From an engineering perspective, this is the desirable approach to building secure compilers.
Improvement of the ocean pollutant transport model by using the surface spline interpolation

Abstract The independent point scheme (IPS) is applied to inverting initial condition with the adjoint method for the ocean pollutant transport model in this work. As an improvement, the linear Cressman interpolation is removed and the surface spline interpolation is implemented in the IPS. A series of numerical experiments are carried out to test and compare the improved IPS. And experiment results show that through applying the improved IPS, what is further reduced is mean absolute errors between simulation results and observations. Moreover, the inverted distributions are more smooth, accurate and reasonable. In addition, the application of improved IPS also reduces the variables that need to be inverted and promotes the computational efficiency. By these numerical experiment results, it is demonstrated that the combination of improved IPS and adjoint method can be used for the inversion of initial conditions and parameters estimation more effectively and reliably.
Towards Optimal Neural Networks: the Role of Sample Splitting in Hyperparameter Selection

When artificial neural networks have demonstrated exceptional practical success in a variety of domains, investigations into their theoretical characteristics, such as their approximation power, statistical properties, and generalization performance, have concurrently made significant strides. In this paper, we construct a novel theory for understanding the effectiveness of neural networks, which offers a perspective distinct from prior research. Specifically, we explore the rationale underlying a common practice during the construction of neural network models: sample splitting. Our findings indicate that the optimal hyperparameters derived from sample splitting can enable a neural network model that asymptotically minimizes the prediction risk. We conduct extensive experiments across different application scenarios and network architectures, and the results manifest our theory's effectiveness.