Miracle-dz/UniFAR-FTUs huggingface dataset:
Task-Oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10

This paper summarizes our contributions to the document-grounded dialog tasks at the 9th and 10th Dialog System Technology Challenges (DSTC9 and DSTC10). In both iterations the task consists of three subtasks: first detect whether the current turn is knowledge seeking, second select a relevant knowledge document, and third generate a response grounded on the selected document. For DSTC9 we proposed different approaches to make the selection task more efficient. The best method, Hierarchical Selection, actually improves the results compared to the original baseline and gives a speedup of 24x. In the DSTC10 iteration of the task, the challenge was to adapt systems trained on written dialogs to perform well on noisy automatic speech recognition transcripts. Therefore, we proposed data augmentation techniques to increase the robustness of the models as well as methods to adapt the style of generated responses to fit well into the proceeding dialog. Additionally, we proposed a noisy channel model that allows for increasing the factuality of the generated responses. In addition to summarizing our previous contributions, in this work, we also report on a few small improvements and reconsider the automatic evaluation metrics for the generation task which have shown a low correlation to human judgments.
Label Dependencies-aware Set Prediction Networks for Multi-label Text Classification

Multi-label text classification involves extracting all relevant labels from a sentence. Given the unordered nature of these labels, we propose approaching the problem as a set prediction task. To address the correlation between labels, we leverage Graph Convolutional Networks and construct an adjacency matrix based on the statistical relations between labels. Additionally, we enhance recall ability by applying the Bhattacharyya distance to the output distributions of the set prediction networks. We evaluate the effectiveness of our approach on two multi-label datasets and demonstrate its superiority over previous baselines through experimental results.
Electron-phonon interactions in LuH$_2$, LuH$_3$, and LuN

This paper presents the calculation results of electron-phonon interactions within the LuH$_2$, LuH$_3$, and LuN systems under 0 GPa and 10 GPa via density functional theory at the GGA-PBE level. The purpose of this work is to provide useful data that may be of the interests of the superconducting community as it was reported that the Lu-H-N compound is likely to be a room-temperature superconductor under 1 GPa [Nature, 615, 244 (2023)].
Predictive Incrementality by Experimentation (PIE) for Ad Measurement

We present a novel approach to causal measurement for advertising, namely to use exogenous variation in advertising exposure (RCTs) for a subset of ad campaigns to build a model that can predict the causal effect of ad campaigns that were run without RCTs. This approach -- Predictive Incrementality by Experimentation (PIE) -- frames the task of estimating the causal effect of an ad campaign as a prediction problem, with the unit of observation being an RCT itself. In contrast, traditional causal inference approaches with observational data seek to adjust covariate imbalance at the user level. A key insight is to use post-campaign features, such as last-click conversion counts, that do not require an RCT, as features in our predictive model. We find that our PIE model recovers RCT-derived incremental conversions per dollar (ICPD) much better than the program evaluation approaches analyzed in Gordon et al. (forthcoming). The prediction errors from the best PIE model are 48%, 42%, and 62% of the RCT-based average ICPD for upper-, mid-, and lower-funnel conversion outcomes, respectively. In contrast, across the same data, the average prediction error of stratified propensity score matching exceeds 491%, and that of double/debiased machine learning exceeds 2,904%. Using a decision-making framework inspired by industry, we show that PIE leads to different decisions compared to RCTs for only 6% of upper-funnel, 7% of mid-funnel, and 13% of lower-funnel outcomes. We conclude that PIE could enable advertising platforms to scale causal ad measurement by extrapolating from a limited number of RCTs to a large set of non-experimental ad campaigns.
Kinematic Earthquake Sequences on Geometrically Complex Faults

Computational earthquake sequence models provide generative estimates of the time, location, and size of synthetic seismic events that can be compared with observed earthquake histories and assessed as rupture forecasts. Here we describe a three-dimensional probabilistic earthquake sequence model that produces slip event time series constrained across geometrically complex non-planar fault systems. This model is kinematic in nature, integrating the time evolution of geometric moment accumulation and release with empirical earthquake scaling laws. The temporal probability of event occurrence is determined from the time history of geometric moment integrated with short-term Omori-style rate decay following each earthquake achieving long-term time-averaged moment balance. Similarly, the net geometric moment monotonically controls the probability of event localization, and seismic events release geometric moment with spatially heterogenous slip on three-dimensional non-planar fault surfaces. We use this model to generate a synthetic earthquake sequence on the Nankai subduction zone over a 1,250-year-long interval, including 700+ $\MW{=}5.5{-}8.5$ coseismic events, with decadal-to-centennial scale quiescent intervals quasi-periodic great earthquake clusters followed by aftershock sequences.
Functional normalizations for the branching process with infinite mean

It is well known that, in a Bienaymé-Galton–Watson process (Zn ) with 1 < m = EZ 1 < ∞ and EZ 1 log Z 1 <∞, the sequence of random variables Znm –n converges a.s. to a non–degenerate limit. When m =∞, an analogous result holds: for any 0< α < 1, it is possible to find functions U such that α n U (Zn ) converges a.s. to a non-degenerate limit. In this paper, some sufficient conditions, expressed in terms of the probability generating function of Z 1 and of its distribution function, are given under which a particular pair (α, U) is appropriate for (Zn ). The most stringent set of conditions reduces, when U (x) x, to the requirements EZ 1 = 1/α, EZ 1 log Z 1 <∞.
End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs

We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) hardware. Our approach leverages Hessian-aware quantization (HAWQ) of NNs, the Quantized Open Neural Network Exchange (QONNX) intermediate representation, and the hls4ml tool flow for transpiling NNs into FPGA and ASIC firmware. This makes efficient NN implementations in hardware accessible to nonexperts, in a single open-sourced workflow that can be deployed for real-time machine learning applications in a wide range of scientific and industrial settings. We demonstrate the workflow in a particle physics application involving trigger decisions that must operate at the 40 MHz collision rate of the CERN Large Hadron Collider (LHC). Given the high collision rate, all data processing must be implemented on custom ASIC and FPGA hardware within a strict area and latency. Based on these constraints, we implement an optimized mixed-precision NN classifier for high-momentum particle jets in simulated LHC proton-proton collisions.
On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence

Large pre-trained models, also known as foundation models (FMs), are trained in a task-agnostic manner on large-scale data and can be adapted to a wide range of downstream tasks by fine-tuning, few-shot, or even zero-shot learning. Despite their successes in language and vision tasks, we have yet seen an attempt to develop foundation models for geospatial artificial intelligence (GeoAI). In this work, we explore the promises and challenges of developing multimodal foundation models for GeoAI. We first investigate the potential of many existing FMs by testing their performances on seven tasks across multiple geospatial subdomains including Geospatial Semantics, Health Geography, Urban Geography, and Remote Sensing. Our results indicate that on several geospatial tasks that only involve text modality such as toponym recognition, location description recognition, and US state-level/county-level dementia time series forecasting, these task-agnostic LLMs can outperform task-specific fully-supervised models in a zero-shot or few-shot learning setting. However, on other geospatial tasks, especially tasks that involve multiple data modalities (e.g., POI-based urban function classification, street view image-based urban noise intensity classification, and remote sensing image scene classification), existing foundation models still underperform task-specific models. Based on these observations, we propose that one of the major challenges of developing a FM for GeoAI is to address the multimodality nature of geospatial tasks. After discussing the distinct challenges of each geospatial data modality, we suggest the possibility of a multimodal foundation model which can reason over various types of geospatial data through geospatial alignments. We conclude this paper by discussing the unique risks and challenges to develop such a model for GeoAI.
Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments

The localization of objects is a crucial task in various applications such as robotics, virtual and augmented reality, and the transportation of goods in warehouses. Recent advances in deep learning have enabled the localization using monocular visual cameras. While structure from motion (SfM) predicts the absolute pose from a point cloud, absolute pose regression (APR) methods learn a semantic understanding of the environment through neural networks. However, both fields face challenges caused by the environment such as motion blur, lighting changes, repetitive patterns, and feature-less structures. This study aims to address these challenges by incorporating additional information and regularizing the absolute pose using relative pose regression (RPR) methods. RPR methods suffer under different challenges, i.e., motion blur. The optical flow between consecutive images is computed using the Lucas-Kanade algorithm, and the relative pose is predicted using an auxiliary small recurrent convolutional network. The fusion of absolute and relative poses is a complex task due to the mismatch between the global and local coordinate systems. State-of-the-art methods fusing absolute and relative poses use pose graph optimization (PGO) to regularize the absolute pose predictions using relative poses. In this work, we propose recurrent fusion networks to optimally align absolute and relative pose predictions to improve the absolute pose prediction. We evaluate eight different recurrent units and construct a simulation environment to pre-train the APR and RPR networks for better generalized training. Additionally, we record a large database of different scenarios in a challenging large-scale indoor environment that mimics a warehouse with transportation robots. We conduct hyperparameter searches and experiments to show the effectiveness of our recurrent fusion method compared to PGO.
Predicting the Fracture Propensity of Amorphous Silica using Molecular Dynamics Simulations and Machine Learning

Amorphous silica ($a-SiO_2$) is a widely used inorganic material. Interestingly, the relationship between the local atomic structures of $a-SiO_2$ and their effects on ductility and fracture is seldom explored. Here, we combine large-scale molecular dynamics simulations and machine learning methods to examine the molecular deformations and fracture mechanisms of $a-SiO_2$. By quenching at high pressures, we demonstrate that densifying $a-SiO_2$ increases the ductility and toughness. Through theoretical analysis and simulation results, we find that changes in local bonding topologies greatly facilitate energy dissipation during plastic deformation, particularly if the coordination numbers decrease. The appearance of fracture can then be accurately located based on the spatial distribution of the atoms. We further observe that the static unstrained structure encodes the propensity for local atomic coordination to change during applied strain, hence a distinct connection can be made between the initial atomic configurations before loading and the final far-from-equilibrium atomic configurations upon fracture. These results are essential for understanding how atomic arrangements strongly influence the mechanical properties and structural features in amorphous solids and will be useful in atomistic design of functional materials.
Optimal measure transformation problems

We introduce an optimal measure transformation problem for zero coupon bond prices based on dynamic relative entropy of probability measures. In the default-free case we prove the equivalence of the optimal measure transformation problem and an optimal stochastic control problem of Gombani and Runggaldier (Math. Financ. 23(4):659-686, 2013) for bond prices. We also consider the optimal measure transformation problem for defaultable bonds, futures contracts, and forward contracts. We provide financial interpretations of the optimal measure transformation problems in terms of the maximization of returns subject to a relative entropy penalty term. In general the solution of the optimal measure transformation problem is characterized by the solution of certain decoupled nonlinear forward-backward stochastic differential equations (FBSDEs). In specific classes of models we show how these FBSDEs can be solved explicitly or at least numerically.
CornerFormer: Boosting Corner Representation for Fine-Grained Structured Reconstruction

Structured reconstruction is a non-trivial dense prediction problem, which extracts structural information (\eg, building corners and edges) from a raster image, then reconstructs it to a 2D planar graph accordingly. Compared with common segmentation or detection problems, it significantly relays on the capability that leveraging holistic geometric information for structural reasoning. Current transformer-based approaches tackle this challenging problem in a two-stage manner, which detect corners in the first model and classify the proposed edges (corner-pairs) in the second model. However, they separate two-stage into different models and only share the backbone encoder. Unlike the existing modeling strategies, we present an enhanced corner representation method: 1) It fuses knowledge between the corner detection and edge prediction by sharing feature in different granularity; 2) Corner candidates are proposed in four heatmap channels w.r. t its direction. Both qualitative and quantitative evaluations demonstrate that our proposed method can better reconstruct fine-grained structures, such as adjacent corners and tiny edges. Consequently, it outperforms the state-of-the-art model by +1.9\%@F-1 on Corner and +3.0\%@F-1 on Edge.
Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction

Integrating whole-slide images (WSIs) and bulk transcriptomics for predicting patient survival can improve our understanding of patient prognosis. However, this multimodal task is particularly challenging due to the different nature of these data: WSIs represent a very high-dimensional spatial description of a tumor, while bulk transcriptomics represent a global description of gene expression levels within that tumor. In this context, our work aims to address two key challenges: (1) how can we tokenize transcriptomics in a semantically meaningful and interpretable way? , and (2) how can we capture dense multimodal interactions between these two modalities? Specifically, we propose to learn biological pathway tokens from transcriptomics that can encode specific cellular functions. Together with histology patch tokens that encode the different morphological patterns in the WSI, we argue that they form appropriate reasoning units for downstream interpretability analyses. We propose fusing both modalities using a memory-efficient multimodal Transformer that can model interactions between pathway and histology patch tokens. Our proposed model, SURVPATH, achieves state-of-the-art performance when evaluated against both unimodal and multimodal baselines on five datasets from The Cancer Genome Atlas. Our interpretability framework identifies key multimodal prognostic factors, and, as such, can provide valuable insights into the interaction between genotype and phenotype, enabling a deeper understanding of the underlying biological mechanisms at play. We make our code public at: https://github.com/ajv012/SurvPath.
Heisenberg-Langevin approach to driven superradiance

We present an analytical approach for the study of driven Dicke superradiance based on a Heisenberg-Langevin formulation. We calculate the steady-state fluctuations of both the atomic-spin and the light-field operators. While the atoms become entangled below a critical drive, exhibiting spin squeezing, we show that the radiated light is in a classical-like coherent state whose amplitude and spectrum are identical to those of the incident driving field. Therefore, the nonlinear atomic system scatters light as a linear classical scatterer. Our results are consistent with the recent theory of coherently radiating spin states. The presented Heisenberg-Langevin approach should be simple to generalize for treating superradiance beyond the permutation-symmetric Dicke model.
Cross-Entropy Loss Functions: Theoretical Analysis and Applications

Cross-entropy is a widely used loss function in applications. It coincides with the logistic loss applied to the outputs of a neural network, when the softmax is used. But, what guarantees can we rely on when using cross-entropy as a surrogate loss? We present a theoretical analysis of a broad family of loss functions, comp-sum losses, that includes cross-entropy (or logistic loss), generalized cross-entropy, the mean absolute error and other cross-entropy-like loss functions. We give the first $H$-consistency bounds for these loss functions. These are non-asymptotic guarantees that upper bound the zero-one loss estimation error in terms of the estimation error of a surrogate loss, for the specific hypothesis set $H$ used. We further show that our bounds are tight. These bounds depend on quantities called minimizability gaps. To make them more explicit, we give a specific analysis of these gaps for comp-sum losses. We also introduce a new family of loss functions, smooth adversarial comp-sum losses, that are derived from their comp-sum counterparts by adding in a related smooth term. We show that these loss functions are beneficial in the adversarial setting by proving that they admit $H$-consistency bounds. This leads to new adversarial robustness algorithms that consist of minimizing a regularized smooth adversarial comp-sum loss. While our main purpose is a theoretical analysis, we also present an extensive empirical analysis comparing comp-sum losses. We further report the results of a series of experiments demonstrating that our adversarial robustness algorithms outperform the current state-of-the-art, while also achieving a superior non-adversarial accuracy.
Towards Learning and Verifying Maximal Neural Lyapunov Functions

The search for Lyapunov functions is a crucial task in the analysis of nonlinear systems. In this paper, we present a physics-informed neural network (PINN) approach to learning a Lyapunov function that is nearly maximal for a given stable set. A Lyapunov function is considered nearly maximal if its sub-level sets can be made arbitrarily close to the boundary of the domain of attraction. We use Zubov's equation to train a maximal Lyapunov function defined on the domain of attraction. Additionally, we propose conditions that can be readily verified by satisfiability modulo theories (SMT) solvers for both local and global stability. We provide theoretical guarantees on the existence of maximal Lyapunov functions and demonstrate the effectiveness of our computational approach through numerical examples.
Generic elasticity of thermal, under-constrained systems

Athermal (i.e. zero-temperature) under-constrained systems are typically floppy, but they can be rigidified by the application of external strain, which is theoretically well understood. Here and in the companion paper, we extend this theory to finite temperatures for a very broad class of under-constrained systems. In the vicinity of the athermal transition point, we derive from first principles expressions for elastic properties such as isotropic tension $t$ and shear modulus $G$ on temperature $T$, isotropic strain $\varepsilon$, and shear strain $\gamma$, which we confirm numerically. These expressions contain only three phenomenological parameters, entropic rigidity $\kappa_S$, energetic rigidity $\kappa_E$, and a parameter $b_\varepsilon$ describing the interaction between isotropic and shear strain. Our results imply that in under-constrained systems, entropic and energetic rigidity interact like two springs in series. This also allows for a simple explanation of the previously numerically observed scaling relation $t\sim G\sim T^{1/2}$ at $\varepsilon=\gamma=0$. Our work unifies the physics of systems as diverse as polymer fibers&networks, membranes, and vertex models for biological tissues.
Hierarchical network structure as the source of power-law frequency spectra (state-trait continua) in living and non-living systems: how physical traits and personalities emerge from first principles in biophysics

What causes organisms to have different body plans and personalities? In this paper, we address this question by looking at universal principles that govern the morphology and behavior of living systems. Living systems invariably display a small world network structure in which many smaller clusters are nested within fewer larger ones, producing a fractal-like structure with a ‘power-law’ cluster size distribution. Apart from their network structure, the dynamics of living systems shows fractal-like qualities: the timeseries of inner message passing and overt behavior contain high frequencies or ‘states’ (treble) that are nested within lower frequencies or ‘traits’ (bass). This produces a power-law frequency spectrum, which is known as a ‘state-trait continuum’ in the behavioral sciences. Here, we argue that the nested modular (power-law) dynamics of living systems results from their nested modular (power-law) network structure: organisms ‘vertically encode’ the deep spatiotemporal structure of their environments, to the effect that high frequencies (states) are produced by many small clusters at the base of a nested-modular hierarchy and lower frequencies (traits) are produced by fewer larger clusters at its top. These include physical traits (body plans) as well as behavioral traits (e.g. personalities). Nested-modular structure causes higher frequencies to be embedded in lower frequencies, producing power-law dynamics. At the heart of such dynamics lies the need for efficient energy dissipation through networks of coupled oscillators (granular media), which also governs the dynamics of non-living systems (e.q. earthquake dynamics, stock market fluctuations). Thus, we provide a single explanation for power-law frequency spectra in both living and non-living systems. If hierarchical structure indeed produces hierarchical dynamics, the development (e.g. during maturation) and collapse (e.g. during disease) of hierarchical structure should leave specific traces in power-law frequency spectra that may serve as early warning signs to system failure. The applications of this idea range from embryology and personality psychology to sociology, evolutionary biology and clinical medicine.