Miracle-dz/UniFAR-FTUs huggingface dataset:
"One-size-fits-all"? Observations and Expectations of NLG Systems Across Identity-Related Language Features

Fairness-related assumptions about what constitutes appropriate NLG system behaviors range from invariance, where systems are expected to respond identically to social groups, to adaptation, where responses should instead vary across them. We design and conduct five case studies, in which we perturb different types of identity-related language features (names, roles, locations, dialect, and style) in NLG system inputs to illuminate tensions around invariance and adaptation. We outline people's expectations of system behaviors, and surface potential caveats of these two contrasting yet commonly-held assumptions. We find that motivations for adaptation include social norms, cultural differences, feature-specific information, and accommodation; motivations for invariance include perspectives that favor prescriptivism, view adaptation as unnecessary or too difficult for NLG systems to do appropriately, and are wary of false assumptions. Our findings highlight open challenges around defining what constitutes fair NLG system behavior.
Comparison of Unscented Kalman Filter Design for Agricultural Anaerobic Digestion Model

Dynamic operation of biological processes, such as anaerobic digestion (AD), requires reliable process monitoring to guarantee stable operating conditions at all times. Unscented Kalman filters (UKF) are an established tool for nonlinear state estimation, and there exist numerous variants of UKF implementations, treating state constraints, improvements of numerical performance and different noise scenarios. So far, however, a unified comparison of proposed methods emphasizing the algorithmic details is lacking. The present study thus examines multiple unconstrained and constrained UKF variants, addresses aspects crucial for direct implementation and applies them to a simplified AD model. The constrained UKF considering additive noise delivered the most accurate state estimations. The long run time of the underlying optimization could be vastly reduced through pre-calculated gradients and Hessian of the associated cost function, as well as by reformulation of the cost function as a quadratic program. However, unconstrained UKF variants showed lower run times and competitive estimation accuracy. This study provides useful advice to practitioners working with nonlinear Kalman filters by paying close attention to algorithmic details and modifications crucial for successful implementation.
Course Correcting Koopman Representations

Koopman representations aim to learn features of nonlinear dynamical systems (NLDS) which lead to linear dynamics in the latent space. Theoretically, such features can be used to simplify many problems in modeling and control of NLDS. In this work we study autoencoder formulations of this problem, and different ways they can be used to model dynamics, specifically for future state prediction over long horizons. We discover several limitations of predicting future states in the latent space and propose an inference-time mechanism, which we refer to as Periodic Reencoding, for faithfully capturing long term dynamics. We justify this method both analytically and empirically via experiments in low and high dimensional NLDS.
Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks

Adversarial training has shown promise in building robust models against adversarial examples. A major drawback of adversarial training is the computational overhead introduced by the generation of adversarial examples. To overcome this limitation, adversarial training based on single-step attacks has been explored. Previous work improves the single-step adversarial training from different perspectives, e.g., sample initialization, loss regularization, and training strategy. Almost all of them treat the underlying model as a black box. In this work, we propose to exploit the interior building blocks of the model to improve efficiency. Specifically, we propose to dynamically sample lightweight subnetworks as a surrogate model during training. By doing this, both the forward and backward passes can be accelerated for efficient adversarial training. Besides, we provide theoretical analysis to show the model robustness can be improved by the single-step adversarial training with sampled subnetworks. Furthermore, we propose a novel sampling strategy where the sampling varies from layer to layer and from iteration to iteration. Compared with previous methods, our method not only reduces the training cost but also achieves better model robustness. Evaluations on a series of popular datasets demonstrate the effectiveness of the proposed FB-Better. Our code has been released at https://github.com/jiaxiaojunQAQ/FP-Better.
Topological constraints on general relativistic galaxies: Exploring novel conical singularity networks

The van Stockum-Bonner class of spacetimes can be interpreted as fully general relativistic models for rigidly rotating disc galaxies. Frame-dragging effects in these geometries demand a recalibration of the dark matter content relative to models based on Newtonian gravity. We investigate the previously overlooked topological structure of these spacetimes, in relation to the viability of fully general relativistic galaxy toy models. We discuss the appropriate boundary conditions for these solutions to model disc galaxies. For this class of spacetimes, we show the existence of a network of quasi-regular singularities along the rotation axis of the galaxies. The existence of such novel conical defect structures further restricts the physical viability of the van Stockum-Bonner class. Unwinding these issues is key to avoiding pathologies in future fully general relativistic modelling of alternative to dark matter.
LGR-MPC: A user-friendly software based on Legendre-Gauss-Radau pseudo spectral method for solving Model Predictive Control problems

Active components, such as actuators, constitute a fundamental aspect of engineering systems, affording the freedom to shape system behavior as desired. However, this capability necessitates energy consumption, primarily in the form of electricity. Thus, a trade-off emerges between energy usage and desired outcomes. While open-loop optimal control methods strive for efficiency, practical implementation is hampered by disturbances and model discrepancies, underscoring the need for closed-loop controllers. The Proportional- Integral-Derivative (PID) controller is widely favored in industry due to its simplicity, despite sub-optimal responses in many cases. To bridge this gap, Model Predictive Control (MPC) offers a solution, yet its complexity limits its broad applicability. This paper introduces user-friendly Python-based MPC software, enabling easy access to MPC. The effectiveness of this software is demonstrated through multiple examples, including those with a known analytical solution.
Empowering Distributed Solutions in Renewable Energy Systems and Grid Optimization

This study delves into the shift from centralized to decentralized approaches in the electricity industry, with a particular focus on how machine learning (ML) advancements play a crucial role in empowering renewable energy sources and improving grid management. ML models have become increasingly important in predicting renewable energy generation and consumption, utilizing various techniques like artificial neural networks, support vector machines, and decision trees. Furthermore, data preprocessing methods, such as data splitting, normalization, decomposition, and discretization, are employed to enhance prediction accuracy. The incorporation of big data and ML into smart grids offers several advantages, including heightened energy efficiency, more effective responses to demand, and better integration of renewable energy sources. Nevertheless, challenges like handling large data volumes, ensuring cybersecurity, and obtaining specialized expertise must be addressed. The research investigates various ML applications within the realms of solar energy, wind energy, and electric distribution and storage, illustrating their potential to optimize energy systems. To sum up, this research demonstrates the evolving landscape of the electricity sector as it shifts from centralized to decentralized solutions through the application of ML innovations and distributed decision-making, ultimately shaping a more efficient and sustainable energy future.
Uniform ergodicity and the one-sided ergodic Hilbert transform

Let $T$ be a bounded linear operator on a Banach space $X$ satisfying $\|T^n\|/n \to 0$. We prove that $T$ is uniformly ergodic if and only if the one-sided ergodic Hilbert transform $H_Tx:= \lim_{n\to\infty} \sum_{k=1}^n k^{-1}T^k x$ converges for every $x \in \overline{(I-T)X}$. When $T$ is power-bounded (or more generally $(C,\alpha)$ bounded for some $0<\alpha<1$), then $T$ is uniformly ergodic if and only if the domain of $H_T$ equals $(I-T)X$. We then study rotational uniform ergodicity -- uniform ergodicity of every $\lambda T$ with $|\lambda|=1$, and connect it to convergence of the rotated one-sided ergodic Hilbert transform, $H_{\lambda T}x$. In the Appendix we prove that positive isometries with finite-dimensional fixed space on infinite-dimensional Banach lattices are never uniformly ergodic. In particular, the Koopman operators of ergodic, even non-invertible, probability preserving transformations on standard spaces are never uniformly ergodic.
Rigidity of Curvature Bounds of Quotient Spaces Of Isometric Actions

Let $G\curvearrowright M$ be an isometric action of a Lie Group on a complete orientable Riemannian manifold. We disintegrate absolutely continuous measures with respect to the volume measure of $M$ along the principal orbits of $G\curvearrowright M$ and define a functional on the probability measures with support on the principal orbits of the action to further prove that the convexity properties of this functional guarantees necessary and sufficient conditions to the Ricci curvature of $M$ to be bound below by a given real number $K$.
Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiency

We consider Bayesian optimization using Gaussian Process models, also referred to as kernel-based bandit optimization. We study the methodology of exploring the domain using random samples drawn from a distribution. We show that this random exploration approach achieves the optimal error rates. Our analysis is based on novel concentration bounds in an infinite dimensional Hilbert space established in this work, which may be of independent interest. We further develop an algorithm based on random exploration with domain shrinking and establish its order-optimal regret guarantees under both noise-free and noisy settings. In the noise-free setting, our analysis closes the existing gap in regret performance and thereby resolves a COLT open problem. The proposed algorithm also enjoys a computational advantage over prevailing methods due to the random exploration that obviates the expensive optimization of a non-convex acquisition function for choosing the query points at each iteration.
A Spline-Based Collocation Method for Stokes and Navier-Stokes equations

In the paper, we propose a collocation method based on multivariate polynomial splines over triangulation or tetrahedralization for solving Stokes and Navier-Stokes equations. We start with a detailed explanation of the method for the Stokes equation and then extend the study to the Navier-Stokes equations. We shall show that the numerical solution can approximate the exact PDE solution very well over several domains. Then we present several numerical experimental results to demonstrate the performance of the method over the 2D and 3D settings. Also, we apply the IPBM method to our method to find the solution over several curved domains effectively. In addition, we present a comparison with the existing multivariate spline methods in \cite{AL02} and several existing methods to show that the new method produces a similar and sometimes more accurate approximation in a more efficient fashion.
Characterizing Mechanisms for Factual Recall in Language Models

Language Models (LMs) often must integrate facts they memorized in pretraining with new information that appears in a given context. These two sources can disagree, causing competition within the model, and it is unclear how an LM will resolve the conflict. On a dataset that queries for knowledge of world capitals, we investigate both distributional and mechanistic determinants of LM behavior in such situations. Specifically, we measure the proportion of the time an LM will use a counterfactual prefix (e.g.,"The capital of Poland is London") to overwrite what it learned in pretraining ("Warsaw"). On Pythia and GPT2, the training frequency of both the query country ("Poland") and the in-context city ("London") highly affect the models' likelihood of using the counterfactual. We then use head attribution to identify individual attention heads that either promote the memorized answer or the in-context answer in the logits. By scaling up or down the value vector of these heads, we can control the likelihood of using the in-context answer on new data. This method can increase the rate of generating the in-context answer to 88\% of the time simply by scaling a single head at runtime. Our work contributes to a body of evidence showing that we can often localize model behaviors to specific components and provides a proof of concept for how future methods might control model behavior dynamically at runtime.
Multimodal Representations for Teacher-Guided Compositional Visual Reasoning

Neural Module Networks (NMN) are a compelling method for visual question answering, enabling the translation of a question into a program consisting of a series of reasoning sub-tasks that are sequentially executed on the image to produce an answer. NMNs provide enhanced explainability compared to integrated models, allowing for a better understanding of the underlying reasoning process. To improve the effectiveness of NMNs we propose to exploit features obtained by a large-scale cross-modal encoder. Also, the current training approach of NMNs relies on the propagation of module outputs to subsequent modules, leading to the accumulation of prediction errors and the generation of false answers. To mitigate this, we introduce an NMN learning strategy involving scheduled teacher guidance. Initially, the model is fully guided by the ground-truth intermediate outputs, but gradually transitions to an autonomous behavior as training progresses. This reduces error accumulation, thus improving training efficiency and final performance.We demonstrate that by incorporating cross-modal features and employing more effective training techniques for NMN, we achieve a favorable balance between performance and transparency in the reasoning process.
Coded Computing for Half-Duplex Wireless Distributed Computing Systems via Interference Alignment

Distributed computing frameworks such as MapReduce and Spark are often used to process large-scale data computing jobs. In wireless scenarios, exchanging data among distributed nodes would seriously suffer from the communication bottleneck due to limited communication resources such as bandwidth and power. To address this problem, we propose a coded parallel computing (CPC) scheme for distributed computing systems where distributed nodes exchange information over a half-duplex wireless interference network. The CPC scheme achieves the multicast gain by utilizing coded computing to multicast coded symbols {intended to} multiple receiver nodes and the cooperative transmission gain by allowing multiple {transmitter} nodes to jointly deliver messages via interference alignment. To measure communication performance, we apply the widely used latency-oriented metric: \emph{normalized delivery time (NDT)}. It is shown that CPC can significantly reduce the NDT by jointly exploiting the parallel transmission and coded multicasting opportunities. Surprisingly, when $K$ tends to infinity and the computation load is fixed, CPC approaches zero NDT while all state-of-the-art schemes achieve positive values of NDT. Finally, we establish an information-theoretic lower bound for the NDT-computation load trade-off over \emph{half-duplex} network, and prove our scheme achieves the minimum NDT within a multiplicative gap of $3$, i.e., our scheme is order optimal.
Euclid preparation. TBD. Forecast impact of super-sample covariance on 3x2pt analysis with Euclid

Deviations from Gaussianity in the distribution of the fields probed by large-scale structure surveys generate additional terms in the data covariance matrix, increasing the uncertainties in the measurement of the cosmological parameters. Super-sample covariance (SSC) is among the largest of these non-Gaussian contributions, with the potential to significantly degrade constraints on some of the parameters of the cosmological model under study -- especially for weak lensing cosmic shear. We compute and validate the impact of SSC on the forecast uncertainties on the cosmological parameters for the Euclid photometric survey, obtained with a Fisher matrix analysis, both considering the Gaussian covariance alone and adding the SSC term -- computed through the public code PySSC. The photometric probes are considered in isolation and combined in the `3$\times$2pt' analysis. We find the SSC impact to be non-negligible -- halving the Figure of Merit of the dark energy parameters ($w_0$, $w_a$) in the 3$\times$2pt case and substantially increasing the uncertainties on $\Omega_{{\rm m},0}, w_0$, and $\sigma_8$ for cosmic shear; photometric galaxy clustering, on the other hand, is less affected due to the lower probe response. The relative impact of SSC does not show significant changes under variations of the redshift binning scheme, while it is smaller for weak lensing when marginalising over the multiplicative shear bias nuisance parameters, which also leads to poorer constraints on the cosmological parameters. Finally, we explore how the use of prior information on the shear and galaxy bias changes the SSC impact. Improving shear bias priors does not have a significant impact, while galaxy bias must be calibrated to sub-percent level to increase the Figure of Merit by the large amount needed to achieve the value when SSC is not included.
Molecular modelling of odd viscoelastic fluids

We consider an active, stochastic microscopic model of particles suspended in a fluid and show that the coarse-grained description of this model renders odd viscoelasticity. The model is made up of odd dumbbells, each featuring a robotic device as the bead, which exhibits a particular torque response. We analytically compute the stress-stress correlator and corroborate the results using molecular dynamics simulations. We also provide a unified analytical framework for several experimental and numerical setups designed to elucidate odd effects in fluids.
Physics-Informed with Power-Enhanced Residual Network for Interpolation and Inverse Problems

This paper introduces a novel neural network structure called the Power-Enhancing residual network, designed to improve interpolation capabilities for both smooth and non-smooth functions in 2D and 3D settings. By adding power terms to residual elements, the architecture boosts the network's expressive power. The study explores network depth, width, and optimization methods, showing the architecture's adaptability and performance advantages. Consistently, the results emphasize the exceptional accuracy of the proposed Power-Enhancing residual network, particularly for non-smooth functions. Real-world examples also confirm its superiority over plain neural network in terms of accuracy, convergence, and efficiency. The study also looks at the impact of deeper network. Moreover, the proposed architecture is also applied to solving the inverse Burgers' equation, demonstrating superior performance. In conclusion, the Power-Enhancing residual network offers a versatile solution that significantly enhances neural network capabilities. The codes implemented are available at: \url{https://github.com/CMMAi/ResNet_for_PINN}.
Sparse Bayesian neural networks for regression: Tackling overfitting and computational challenges in uncertainty quantification

Neural networks (NNs) are primarily developed within the frequentist statistical framework. Nevertheless, frequentist NNs lack the capability to provide uncertainties in the predictions, and hence their robustness can not be adequately assessed. Conversely, the Bayesian neural networks (BNNs) naturally offer predictive uncertainty by applying Bayes' theorem. However, their computational requirements pose significant challenges. Moreover, both frequentist NNs and BNNs suffer from overfitting issues when dealing with noisy and sparse data, which render their predictions unwieldy away from the available data space. To address both these problems simultaneously, we leverage insights from a hierarchical setting in which the parameter priors are conditional on hyperparameters to construct a BNN by applying a semi-analytical framework known as nonlinear sparse Bayesian learning (NSBL). We call our network sparse Bayesian neural network (SBNN) which aims to address the practical and computational issues associated with BNNs. Simultaneously, imposing a sparsity-inducing prior encourages the automatic pruning of redundant parameters based on the automatic relevance determination (ARD) concept. This process involves removing redundant parameters by optimally selecting the precision of the parameters prior probability density functions (pdfs), resulting in a tractable treatment for overfitting. To demonstrate the benefits of the SBNN algorithm, the study presents an illustrative regression problem and compares the results of a BNN using standard Bayesian inference, hierarchical Bayesian inference, and a BNN equipped with the proposed algorithm. Subsequently, we demonstrate the importance of considering the full parameter posterior by comparing the results with those obtained using the Laplace approximation with and without NSBL.
Prevalence and prevention of large language model use in crowd work

We show that the use of large language models (LLMs) is prevalent among crowd workers, and that targeted mitigation strategies can significantly reduce, but not eliminate, LLM use. On a text summarization task where workers were not directed in any way regarding their LLM use, the estimated prevalence of LLM use was around 30%, but was reduced by about half by asking workers to not use LLMs and by raising the cost of using them, e.g., by disabling copy-pasting. Secondary analyses give further insight into LLM use and its prevention: LLM use yields high-quality but homogeneous responses, which may harm research concerned with human (rather than model) behavior and degrade future models trained with crowdsourced data. At the same time, preventing LLM use may be at odds with obtaining high-quality responses; e.g., when requesting workers not to use LLMs, summaries contained fewer keywords carrying essential information. Our estimates will likely change as LLMs increase in popularity or capabilities, and as norms around their usage change. Yet, understanding the co-evolution of LLM-based tools and users is key to maintaining the validity of research done using crowdsourcing, and we provide a critical baseline before widespread adoption ensues.