Miracle-dz/UniFAR-FTUs huggingface dataset:
Computationally Efficient and Error Aware Surrogate Construction for Numerical Solutions of Subsurface Flow Through Porous Media

Limiting the injection rate to restrict the pressure below a threshold at a critical location can be an important goal of simulations that model the subsurface pressure between injection and extraction wells. The pressure is approximated by the solution of Darcy's partial differential equation (PDE) for a given permeability field. The subsurface permeability is modeled as a random field since it is known only up to statistical properties. This induces uncertainty in the computed pressure. Solving the PDE for an ensemble of random permeability simulations enables estimating a probability distribution for the pressure at the critical location. These simulations are computationally expensive, and practitioners often need rapid online guidance for real-time pressure management. An ensemble of numerical PDE solutions is used to construct a Gaussian process regression model that can quickly predict the pressure at the critical location as a function of the extraction rate and permeability realization. Our first novel contribution is to identify a sampling methodology for the random environment and matching kernel technology for which fitting the Gaussian process regression model scales as O(n log n) instead of the typical O(n^3) rate in the number of samples n used to fit the surrogate. The surrogate model allows almost instantaneous predictions for the pressure at the critical location as a function of the extraction rate and permeability realization. Our second contribution is a novel algorithm to calibrate the uncertainty in the surrogate model to the discrepancy between the true pressure solution of Darcy's equation and the numerical solution. Although our method is derived for building a surrogate for the solution of Darcy's equation with a random permeability field, the framework broadly applies to solutions of other PDE with random coefficients.
FABULA: Intelligence Report Generation Using Retrieval-Augmented Narrative Construction

Narrative construction is the process of representing disparate event information into a logical plot structure that models an end to end story. Intelligence analysis is an example of a domain that can benefit tremendously from narrative construction techniques, particularly in aiding analysts during the largely manual and costly process of synthesizing event information into comprehensive intelligence reports. Manual intelligence report generation is often prone to challenges such as integrating dynamic event information, writing fine-grained queries, and closing information gaps. This motivates the development of a system that retrieves and represents critical aspects of events in a form that aids in automatic generation of intelligence reports. We introduce a Retrieval Augmented Generation (RAG) approach to augment prompting of an autoregressive decoder by retrieving structured information asserted in a knowledge graph to generate targeted information based on a narrative plot model. We apply our approach to the problem of neural intelligence report generation and introduce FABULA, framework to augment intelligence analysis workflows using RAG. An analyst can use FABULA to query an Event Plot Graph (EPG) to retrieve relevant event plot points, which can be used to augment prompting of a Large Language Model (LLM) during intelligence report generation. Our evaluation studies show that the plot points included in the generated intelligence reports have high semantic relevance, high coherency, and low data redundancy.
BVARs and Stochastic Volatility

Bayesian vector autoregressions (BVARs) are the workhorse in macroeconomic forecasting. Research in the last decade has established the importance of allowing time-varying volatility to capture both secular and cyclical variations in macroeconomic uncertainty. This recognition, together with the growing availability of large datasets, has propelled a surge in recent research in building stochastic volatility models suitable for large BVARs. Some of these new models are also equipped with additional features that are especially desirable for large systems, such as order invariance -- i.e., estimates are not dependent on how the variables are ordered in the BVAR -- and robustness against COVID-19 outliers. Estimation of these large, flexible models is made possible by the recently developed equation-by-equation approach that drastically reduces the computational cost of estimating large systems. Despite these recent advances, there remains much ongoing work, such as the development of parsimonious approaches for time-varying coefficients and other types of nonlinearities in large BVARs.
Reforming Physics Exams Using Openly Accessible Large Isomorphic Problem Banks created with the assistance of Generative AI: an Explorative Study

This paper explores using large isomorphic problem banks to overcome many challenges of traditional exams in large STEM classes, especially the threat of content sharing websites and generative AI to the security of exam items. We first introduce an efficient procedure for creating large numbers of isomorphic physics problems, assisted by the large language model GPT-3 and several other open-source tools. We then propose that if exam items are randomly drawn from large enough problem banks, then giving students open access to problem banks prior to the exam will not dramatically impact students' performance on the exam or lead to wide-spread rote-memorization of solutions. We tested this hypothesis on two mid-term physics exams, comparing students' performance on problems drawn from open isomorphic problem banks to similar transfer problems that were not accessible to students prior to the exam. We found that on both exams, both open bank and transfer problems had the highest difficulty. The differences in percent correct were between 5% to 10%, which is comparable to the differences between different isomorphic versions of the same problem type. Item response theory analysis found that both types of problem have high discrimination (>1.5) with no significant differences. Student performance on open-bank and transfer problems are highly correlated with each other, and the correlations are stronger than average correlations between problems on the exam. Exploratory factor analysis also found that open-bank and transfer problems load on the same factor, and even formed their own factor on the second exam. Those observations all suggest that giving students open access to large isomorphic problem banks only had a small impact on students' performance on the exam but could have significant potential in reforming traditional classroom exams.
Training Image Derivatives: Increased Accuracy and Universal Robustness

Derivative training is a known method that significantly improves the accuracy of neural networks in some low-dimensional applications. In this paper, a similar improvement is obtained for an image analysis problem: reconstructing the vertices of a cube from its image. By training the derivatives with respect to the 6 degrees of freedom of the cube, we obtain 25 times more accurate results for noiseless inputs. The derivatives also offer insight into the robustness problem, which is currently understood in terms of two types of network vulnerabilities. The first type involves small perturbations that dramatically change the output, and the second type relates to substantial image changes that the network erroneously ignores. Defense against each is possible, but safeguarding against both while maintaining the accuracy defies conventional training methods. The first type is analyzed using the network's gradient, while the second relies on human input evaluation, serving as an oracle substitute. For the task at hand, the nearest neighbor oracle can be defined and expanded into Taylor series using image derivatives. This allows for a robustness analysis that unifies both types of vulnerabilities and enables training where accuracy and universal robustness are limited only by network capacity.
Fairness-aware Optimal Graph Filter Design

Graphs are mathematical tools that can be used to represent complex real-world interconnected systems, such as financial markets and social networks. Hence, machine learning (ML) over graphs has attracted significant attention recently. However, it has been demonstrated that ML over graphs amplifies the already existing bias towards certain under-represented groups in various decision-making problems due to the information aggregation over biased graph structures. Faced with this challenge, here we take a fresh look at the problem of bias mitigation in graph-based learning by borrowing insights from graph signal processing. Our idea is to introduce predesigned graph filters within an ML pipeline to reduce a novel unsupervised bias measure, namely the correlation between sensitive attributes and the underlying graph connectivity. We show that the optimal design of said filters can be cast as a convex problem in the graph spectral domain. We also formulate a linear programming (LP) problem informed by a theoretical bias analysis, which attains a closed-form solution and leads to a more efficient fairness-aware graph filter. Finally, for a design whose degrees of freedom are independent of the input graph size, we minimize the bias metric over the family of polynomial graph convolutional filters. Our optimal filter designs offer complementary strengths to explore favorable fairness-utility-complexity tradeoffs. For performance evaluation, we conduct extensive and reproducible node classification experiments over real-world networks. Our results show that the proposed framework leads to better fairness measures together with similar utility compared to state-of-the-art fairness-aware baselines.
ASBART: Accelerated Soft Bayes Additive Regression Trees

Bayes additive regression trees(BART) is a nonparametric regression model which has gained wide-spread popularity in recent years due to its flexibility and high accuracy of estimation. Soft BART,one variation of BART,improves both practically and heoretically on existing Bayesian sum-of-trees models. One bottleneck for Soft BART is its slow speed in the long MCMC loop. Compared to BART,it use more than about 20 times to complete the calculation with the default setting. We proposed a variant of BART named accelerate Soft BART(ASBART). Simulation studies show that the new method is about 10 times faster than the Soft BART with comparable accuracy. Our code is open-source and available at https://github.com/richael008/XSBART.
What's in a Prior? Learned Proximal Networks for Inverse Problems

Proximal operators are ubiquitous in inverse problems, commonly appearing as part of algorithmic strategies to regularize problems that are otherwise ill-posed. Modern deep learning models have been brought to bear for these tasks too, as in the framework of plug-and-play or deep unrolling, where they loosely resemble proximal operators. Yet, something essential is lost in employing these purely data-driven approaches: there is no guarantee that a general deep network represents the proximal operator of any function, nor is there any characterization of the function for which the network might provide some approximate proximal. This not only makes guaranteeing convergence of iterative schemes challenging but, more fundamentally, complicates the analysis of what has been learned by these networks about their training data. Herein we provide a framework to develop learned proximal networks (LPN), prove that they provide exact proximal operators for a data-driven nonconvex regularizer, and show how a new training strategy, dubbed proximal matching, provably promotes the recovery of the log-prior of the true data distribution. Such LPN provide general, unsupervised, expressive proximal operators that can be used for general inverse problems with convergence guarantees. We illustrate our results in a series of cases of increasing complexity, demonstrating that these models not only result in state-of-the-art performance, but provide a window into the resulting priors learned from data.
ÆSOPUS 2.0: Low-temperature Opacities with Solid Grains

In this study we compute the equation of state and Rosseland mean opacity from temperatures of T ≃ 30,000 K down to T ≃ 400 K, pushing the capabilities of the ÆSOPUS code into the regime where solid grains can form. The GGchem code is used to solve the chemistry for temperatures less than ≃3000 K. Atoms, molecules, and dust grains in thermodynamic equilibrium are all included in the equation of state. To incorporate monochromatic atomic and molecular cross sections, an optimized opacity sampling technique is used. The Mie theory is employed to calculate the opacity of 43 grain species. Tables of Rosseland mean opacities for scaled-solar compositions are provided. Based on our computing resources, opacities for other chemical patterns, as well as various grain sizes, porosities, and shapes, can be easily computed upon user request to the corresponding author.
Improving Molecular Properties Prediction Through Latent Space Fusion

Pre-trained Language Models have emerged as promising tools for predicting molecular properties, yet their development is in its early stages, necessitating further research to enhance their efficacy and address challenges such as generalization and sample efficiency. In this paper, we present a multi-view approach that combines latent spaces derived from state-of-the-art chemical models. Our approach relies on two pivotal elements: the embeddings derived from MHG-GNN, which represent molecular structures as graphs, and MoLFormer embeddings rooted in chemical language. The attention mechanism of MoLFormer is able to identify relations between two atoms even when their distance is far apart, while the GNN of MHG-GNN can more precisely capture relations among multiple atoms closely located. In this work, we demonstrate the superior performance of our proposed multi-view approach compared to existing state-of-the-art methods, including MoLFormer-XL, which was trained on 1.1 billion molecules, particularly in intricate tasks such as predicting clinical trial drug toxicity and inhibiting HIV replication. We assessed our approach using six benchmark datasets from MoleculeNet, where it outperformed competitors in five of them. Our study highlights the potential of latent space fusion and feature integration for advancing molecular property prediction. In this work, we use small versions of MHG-GNN and MoLFormer, which opens up an opportunity for further improvement when our approach uses a larger-scale dataset.
Systemic risk approach to mitigate delay cascading in railway networks

In public railway systems, minor disruptions can trigger cascading events that lead to delays in the entire system. Typically, delays originate and propagate because the equipment is blocking ways, operational units are unavailable, or at the wrong place at the needed time. The specific understanding of the origins and processes involved in delay-spreading is still a challenge, even though large-scale simulations of national railway systems are becoming available on a highly detailed scale. Without this understanding, efficient management of delay propagation, a growing concern in some Western countries, will remain impossible. Here, we present a systemic risk-based approach to manage daily delay cascading on national scales. We compute the {\em systemic impact} of every train as the maximum of all delays it could possibly cause due to its interactions with other trains, infrastructure, and operational units. To compute it, we design an effective impact network where nodes are train services and links represent interactions that could cause delays. Our results are not only consistent with highly detailed and computationally intensive agent-based railway simulations but also allow us to pinpoint and identify the causes of delay cascades in detail. The systemic approach reveals structural weaknesses in railway systems whenever shared resources are involved. We use the systemic impact to optimally allocate additional shared resources to the system to reduce delays with minimal costs and effort. The method offers a practical and intuitive solution for delay management by optimizing the effective impact network through the introduction of new cheap local train services.
High order asymptotic preserving and well-balanced schemes for the shallow water equations with source terms

In this study, we investigate the Shallow Water Equations incorporating source terms accounting for Manning friction and a non-flat bottom topology. Our primary focus is on developing and validating numerical schemes that serve a dual purpose: firstly, preserving all steady states within the model, and secondly, maintaining the late-time asymptotic behavior of solutions, which is governed by a diffusion equation and coincides with a long time and stiff friction limit. Our proposed approach draws inspiration from a penalization technique adopted in {\it{[Boscarino et. al, SIAM Journal on Scientific Computing, 2014]}}. By employing an additive implicit-explicit Runge-Kutta method, the scheme can ensure a correct asymptotic behavior for the limiting diffusion equation, without suffering from a parabolic-type time step restriction which often afflicts multiscale problems in the diffusive limit. Numerical experiments are performed to illustrate high order accuracy, asymptotic preserving, and asymptotically accurate properties of the designed schemes.
Augmenting End-to-End Steering Angle Prediction with CAN Bus Data

In recent years, end to end steering prediction for autonomous vehicles has become a major area of research. The primary method for achieving end to end steering was to use computer vision models on a live feed of video data. However, to further increase accuracy, many companies have added data from light detection and ranging (LiDAR) and or radar sensors through sensor fusion. However, the addition of lasers and sensors comes at a high financial cost. In this paper, I address both of these issues by increasing the accuracy of the computer vision models without the increased cost of using LiDAR and or sensors. I achieved this by improving the accuracy of computer vision models by sensor fusing CAN bus data, a vehicle protocol, with video data. CAN bus data is a rich source of information about the vehicle's state, including its speed, steering angle, and acceleration. By fusing this data with video data, the accuracy of the computer vision model's predictions can be improved. When I trained the model without CAN bus data, I obtained an RMSE of 0.02492, while the model trained with the CAN bus data achieved an RMSE of 0.01970. This finding indicates that fusing CAN Bus data with video data can reduce the computer vision model's prediction error by 20% with some models decreasing the error by 80%.
Automatic Sensor-free Affect Detection: A Systematic Literature Review

Emotions and other affective states play a pivotal role in cognition and, consequently, the learning process. It is well-established that computer-based learning environments (CBLEs) that can detect and adapt to students' affective states can enhance learning outcomes. However, practical constraints often pose challenges to the deployment of sensor-based affect detection in CBLEs, particularly for large-scale or long-term applications. As a result, sensor-free affect detection, which exclusively relies on logs of students' interactions with CBLEs, emerges as a compelling alternative. This paper provides a comprehensive literature review on sensor-free affect detection. It delves into the most frequently identified affective states, the methodologies and techniques employed for sensor development, the defining attributes of CBLEs and data samples, as well as key research trends. Despite the field's evident maturity, demonstrated by the consistent performance of the models and the application of advanced machine learning techniques, there is ample scope for future research. Potential areas for further exploration include enhancing the performance of sensor-free detection models, amassing more samples of underrepresented emotions, and identifying additional emotions. There is also a need to refine model development practices and methods. This could involve comparing the accuracy of various data collection techniques, determining the optimal granularity of duration, establishing a shared database of action logs and emotion labels, and making the source code of these models publicly accessible. Future research should also prioritize the integration of models into CBLEs for real-time detection, the provision of meaningful interventions based on detected emotions, and a deeper understanding of the impact of emotions on learning.
Temperature-heat uncertainty relation for quantum thermometry

We investigate the resource theory for temperature estimation. We demonstrate that it is the fluctuation of heat that fundamentally determines temperature precision through the temperature-heat uncertainty relation. Specifically, we find that heat is divided into trajectory heat and correlation heat, which are associated with the heat exchange along thermometer's evolution path and the correlation between the thermometer and the sample, respectively. Based on two type of thermometers, we show that both of these heat terms are resources for enhancing temperature precision. Additionally, we demonstrate that the temperature-heat uncertainty relation is consistent with the well known temperature-energy uncertainty relation in thermodynamics. By clearly distinguishing the resources for enhancing estimation precision, our findings not only explain why various quantum features are crucial for accurate temperature sensing but also provide valuable insights for designing ultrahigh-sensitive quantum thermometers.
Gradual Domain Adaptation: Theory and Algorithms

Unsupervised domain adaptation (UDA) adapts a model from a labeled source domain to an unlabeled target domain in a one-off way. Though widely applied, UDA faces a great challenge whenever the distribution shift between the source and the target is large. Gradual domain adaptation (GDA) mitigates this limitation by using intermediate domains to gradually adapt from the source to the target domain. In this work, we first theoretically analyze gradual self-training, a popular GDA algorithm, and provide a significantly improved generalization bound compared with Kumar et al. (2020). Our theoretical analysis leads to an interesting insight: to minimize the generalization error on the target domain, the sequence of intermediate domains should be placed uniformly along the Wasserstein geodesic between the source and target domains. The insight is particularly useful under the situation where intermediate domains are missing or scarce, which is often the case in real-world applications. Based on the insight, we propose $\textbf{G}$enerative Gradual D$\textbf{O}$main $\textbf{A}$daptation with Optimal $\textbf{T}$ransport (GOAT), an algorithmic framework that can generate intermediate domains in a data-dependent way. More concretely, we first generate intermediate domains along the Wasserstein geodesic between two given consecutive domains in a feature space, then apply gradual self-training to adapt the source-trained classifier to the target along the sequence of intermediate domains. Empirically, we demonstrate that our GOAT framework can improve the performance of standard GDA when the given intermediate domains are scarce, significantly broadening the real-world application scenarios of GDA. Our code is available at https://github.com/yifei-he/GOAT.