Miracle-dz/UniFAR-FTUs huggingface dataset:
Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks

Bayesian optimization (BO) is a powerful approach for optimizing black-box, expensive-to-evaluate functions. To enable a flexible trade-off between the cost and accuracy, many applications allow the function to be evaluated at different fidelities. In order to reduce the optimization cost while maximizing the benefit-cost ratio, in this paper, we propose Batch Multi-fidelity Bayesian Optimization with Deep Auto-Regressive Networks (BMBO-DARN). We use a set of Bayesian neural networks to construct a fully auto-regressive model, which is expressive enough to capture strong yet complex relationships across all the fidelities, so as to improve the surrogate learning and optimization performance. Furthermore, to enhance the quality and diversity of queries, we develop a simple yet efficient batch querying method, without any combinatorial search over the fidelities. We propose a batch acquisition function based on Max-value Entropy Search (MES) principle, which penalizes highly correlated queries and encourages diversity. We use posterior samples and moment matching to fulfill efficient computation of the acquisition function and conduct alternating optimization over every fidelity-input pair, which guarantees an improvement at each step. We demonstrate the advantage of our approach on four real-world hyperparameter optimization applications.
Conservative iterative methods for implicit discretizations of conservation laws

Conservation properties of iterative methods applied to implicit finite volume discretizations of nonlinear conservation laws are analyzed. It is shown that any consistent multistep or Runge-Kutta method is globally conservative. Further, it is shown that Newton's method, Krylov subspace methods and pseudo-time iterations are globally conservative while the Jacobi and Gauss-Seidel methods are not in general. If pseudo-time iterations using an explicit Runge-Kutta method are applied to a locally conservative discretization, then the resulting scheme is also locally conservative. However, the corresponding numerical flux can be inconsistent with the conservation law. We prove an extension of the Lax-Wendroff theorem, which reveals that numerical solutions based on these methods converge to weak solutions of a modified conservation law where the flux function is multiplied by a particular constant. This constant depends on the choice of Runge-Kutta method but is independent of both the conservation law and the discretization. Consistency is maintained by ensuring that this constant equals unity and a strategy for achieving this is presented. Experiments show that this strategy improves the convergence rate of the pseudo-time iterations.
Human-Aware Navigation Planner for Diverse Human-Robot Contexts

As more robots are being deployed into human environments, a human-aware navigation planner needs to handle multiple contexts that occur in indoor and outdoor environments. In this paper, we propose a tunable human-aware robot navigation planner that can handle a variety of humanrobot contexts. We present the architecture of the planner and discuss the features and some implementation details. Then we present a detailed analysis of various simulated humanrobot contexts using the proposed planner along with some quantitative results. Finally, we show the results in a real-world scenario after deploying our system on a real robot.
IQ Collaboratory. III. The Empirical Dust Attenuation Framework—Taking Hydrodynamical Simulations with a Grain of Dust

We present the empirical dust attenuation (EDA) framework—a flexible prescription for assigning realistic dust attenuation to simulated galaxies based on their physical properties. We use the EDA to forward model synthetic observations for three state-of-the-art large-scale cosmological hydrodynamical simulations: SIMBA, IllustrisTNG, and EAGLE. We then compare the optical and UV color–magnitude relations, (g − r) − M r and (far-UV −near-UV) − M r , of the simulations to a M r < − 20 and UV complete Sloan Digital Sky Survey galaxy sample using likelihood-free inference. Without dust, none of the simulations match observations, as expected. With the EDA, however, we can reproduce the observed color–magnitude with all three simulations. Furthermore, the attenuation curves predicted by our dust prescription are in good agreement with the observed attenuation–slope relations and attenuation curves of star-forming galaxies. However, the EDA does not predict star-forming galaxies with low A V since simulated star-forming galaxies are intrinsically much brighter than observations. Additionally, the EDA provides, for the first time, predictions on the attenuation curves of quiescent galaxies, which are challenging to measure observationally. Simulated quiescent galaxies require shallower attenuation curves with lower amplitude than star-forming galaxies. The EDA, combined with forward modeling, provides an effective approach for shedding light on dust in galaxies and probing hydrodynamical simulations. This work also illustrates a major limitation in comparing galaxy formation models: by adjusting dust attenuation, simulations that predict significantly different galaxy populations can reproduce the same UV and optical observations.
Federated Robustness Propagation: Sharing Adversarial Robustness in Federated Learning

Federated learning (FL) emerges as a popular distributed learning schema that learns a model from a set of participating users without requiring raw data to be shared. One major challenge of FL comes from heterogeneity in users, which may have distributionally different (or non-iid ) data and varying computation resources. Just like in centralized learning, FL users also desire model robustness against malicious attackers at test time. Whereas adversarial training (AT) provides a sound solution for centralized learning, extending its usage for FL users has imposed signiﬁcant challenges, as many users may have very limited training data as well as tight computational budgets, to afford the data-hungry and costly AT. In this paper, we study a novel learning setting that propagates adversarial robustness from high-resource users that can afford AT, to those low-resource users that cannot afford it, during the FL process. We show that existing FL techniques cannot effectively propagate adversarial robustness among non-iid users, and propose a simple yet effective propagation approach that transfers robustness through carefully designed batch-normalization statistics. We demonstrate the rationality and effectiveness of our method through extensive experiments. Especially, the proposed method is shown to grant FL remarkable robustness even when only a small portion of users afford AT during learning. Codes will be published upon acceptance.
PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction

Joint extraction of entities and relations from unstructured texts is a crucial task in information extraction. Recent methods achieve considerable performance but still suffer from some inherent limitations, such as redundancy of relation prediction, poor generalization of span-based extraction and inefficiency. In this paper, we decompose this task into three subtasks, Relation Judgement, Entity Extraction and Subject-object Alignment from a novel perspective and then propose a joint relational triple extraction framework based on Potential Relation and Global Correspondence (PRGC). Specifically, we design a component to predict potential relations, which constrains the following entity extraction to the predicted relation subset rather than all relations; then a relation-specific sequence tagging component is applied to handle the overlapping problem between subjects and objects; finally, a global correspondence component is designed to align the subject and object into a triple with low-complexity. Extensive experiments show that PRGC achieves state-of-the-art performance on public benchmarks with higher efficiency and delivers consistent performance gain on complex scenarios of overlapping triples. The source code has been submitted as the supplementary material and will be made publicly available after the blind review.
Dual-view Molecule Pre-training

Inspired by its success in natural language processing and computer vision, pre-training has attracted substantial attention in cheminformatics and bioinformatics, especially for molecule based tasks. A molecule can be represented by either a graph (where atoms are connected by bonds) or a SMILES sequence (where depth-first-search is applied to the molecular graph with specific rules). Existing works on molecule pre-training use either graph representations only or SMILES representations only. In this work, we propose to leverage both the representations and design a new pre-training algorithm, dual-view molecule pre-training (briefly, DMP), that can effectively combine the strengths of both types of molecule representations. The model of DMP consists of two branches: a Transformer branch that takes the SMILES sequence of a molecule as input, and a GNN branch that takes a molecular graph as input. The training of DMP contains three tasks: (1) predicting masked tokens in a SMILES sequence by the Transformer branch, (2) predicting masked atoms in a molecular graph by the GNN branch, and (3) maximizing the consistency between the two high-level representations output by the Transformer and GNN branches separately. After pre-training, we can use either the Transformer branch (this one is recommended according to empirical results), the GNN branch, or both for downstream tasks. DMP is tested on nine molecular property prediction tasks and achieves state-of-the-art performances on seven of them. Furthermore, we test DMP on three retrosynthesis tasks and achieve state-of-the-art results on them.
Sharp complexity asymptotics and topological trivialization for the (p, k) spiked tensor model

Using precise random matrix theory tools and the Kac–Rice formula, we provide sharp O(1) asymptotics for the average number of deep minima of the ( p, k) spiked tensor model. These sharp estimates allow us to prove that, when the signal-to-noise ratio is large enough, the expected number of deep minima is asymptotically finite as N tends to infinity and to establish the occurrence of topological trivialization by showing that this number vanishes when the strength of the signal-to-noise ratio diverges. We also derive an explicit formula for the value of the absolute minimum (the limiting ground state energy) on the N-dimensional sphere, similar to the recent work of Jagannath, Lopatto, and Miolane [Ann. Appl. Probab. 4, 1910–1933 (2020)].
Learning to Generate Code Sketches

Traditional generative models are limited to predicting sequences of terminal tokens. However, ambiguities in the generation task may lead to incorrect outputs. Towards addressing this, we introduce G RAMMFORMER s, transformer-based grammar-guided models that learn (without explicit supervision) to generate sketches — sequences of tokens with holes. Through reinforcement learning, G RAMMFORMER s learn to introduce holes avoiding the generation of incorrect tokens where there is ambiguity in the target task. We train G RAMMFORMER s for statement-level source code completion, i.e. the generation of code snippets given an ambiguous user intent, such as a partial code context. We evaluate G RAMMFORMER s on code completion for C# and Python and show that it generates 10-50% more accurate sketches compared to traditional generative models and 37-50% longer sketches compared to sketch-generating baselines trained with similar techniques.
Moment-Based Parameter Estimation for the Γ-Parameterized TWDP Model

Two-wave with diffuse power (TWDP) is one of the most promising models for description of a small-scale fading effects in the emerging wireless networks. However, its conventional parameterization based on parameters K and Δ is not in line with model’s underlying physical mechanisms. Accordingly, in this paper, we first identified anomalies related to usage of conventional TWDP parameterization in moment-based estimation, showing that the existing Δ-based estimators are unable to provide meaningful estimates in some channel conditions. Then, we derived moment-based estimators of recently introduced physically justified TWDP parameters K and Γ and analyzed their performance through asymptotic variance (AsV) and Cramer–Rao bound (CRB) metrics. Performed analysis has shown that Γ-based estimators managed to overcome all anomalies observed for Δ-based estimators, simultaneously improving the overall moment-based estimation accuracy.
Two-Step Solid-State Synthesis of Ternary Nitride Materials

Ternary nitride materials hold promise for many optical, electronic, and refractory applications; yet, their preparation via solid-state synthesis remains challenging. Often, high pressures or reactive gases are used to manipulate the effective chemical potential of nitrogen, yet these strategies require specialized equipment. Here, we report on a simple two-step synthesis using ion-exchange reactions that yield rocksalt-derived MgZrN2 and Mg2NbN3, as well as layered MgMoN2. All three compounds show almost temperature-independent and weak paramagnetic responses to an applied magnetic field at cryogenic temperatures, indicating phase-pure products. The key to synthesizing these ternary materials is an initial low-temperature step (300–450 °C) to promote Mg-M-N nucleation. The intermediates then are annealed (800–900 °C) to grow crystalline domains of the ternary product. Calorimetry experiments reveal that initial reaction temperatures are determined by phase transitions of reaction precursors, whereas heating directly to high temperatures results in decomposition. These two-step reactions provide a rational guide to material discovery of other bulk ternary nitrides.
Nonparametric Hamiltonian Monte Carlo

Probabilistic programming uses programs to express generative models whose posterior probability is then computed by built-in inference engines. A challenging goal is to develop general purpose inference algorithms that work out-of-the-box for arbitrary programs in a universal probabilistic programming language (PPL). The densities defined by such programs, which may use stochastic branching and recursion, are (in general) nonparametric, in the sense that they correspond to models on an infinite-dimensional parameter space. However standard inference algorithms, such as the Hamiltonian Monte Carlo (HMC) algorithm, target distributions with a fixed number of parameters. This paper introduces the Nonparametric Hamiltonian Monte Carlo (NP-HMC) algorithm which generalises HMC to nonparametric models. Inputs to NP-HMC are a new class of measurable functions called"tree representable", which serve as a language-independent representation of the density functions of probabilistic programs in a universal PPL. We provide a correctness proof of NP-HMC, and empirically demonstrate significant performance improvements over existing approaches on several nonparametric examples.
Engineered dissipation induced entanglement transition in quantum spin chains: From logarithmic growth to area law

Recent theoretical work has shown that the competition between coherent unitary dynamics and stochastic measurements, performed by the environment, along wavefunction trajectories can give rise to transitions in the entanglement scaling. In this work, complementary to these previous studies, we analyze a situation where the role of Hamiltonian and dissipative dynamics is reversed. We consider an engineered dissipation, which stabilizes an entangled phase of a quantum spin$-1/2$ chain, while competing single-particle or interacting Hamiltonian dynamics induce a disentangled phase. Focusing on the single-particle unitary dynamics, we find that the system undergoes an entanglement transition from a logarithmic growth to an area law when the competition ratio between the unitary evolution and the non-unitary dynamics increases. We evidence that the transition manifests itself in state-dependent observables at a finite competition ratio for Hamiltonian and measurement dynamics. On the other hand, it is absent in trajectory-averaged steady-state dynamics, governed by a Lindblad master equation: although purely dissipative dynamics stabilizes an entangled state, for any non-vanishing Hamiltonian contribution the system ends up irremediably in a disordered phase. In addition, a single trajectory analysis reveals that the distribution of the entanglement entropy constitutes an efficient indicator of the transition. Complementarily, we explore the competition of the dissipation with coherent dynamics generated by an interacting Hamiltonian, and demonstrate that the entanglement transition also occurs in this second model. Our results suggest that this type of transition takes place for a broader class of Hamiltonians, underlining its robustness in monitored open quantum many-body systems.
Evolving GANs: When Contradictions Turn into Compliance

Limited availability of labeled-data makes any supervised learning problem challenging. Alternative learning settings like semi-supervised and universum learning alleviate the dependency on labeled data, but still require a large amount of unlabeled data, which may be unavailable or expensive to acquire. GAN-based synthetic data generation methods have recently shown promise by generating synthetic samples to improve task at hand. However, these samples cannot be used for other purposes. In this paper, we propose a GAN game which provides improved discriminator accuracy under limited data settings, while generating realistic synthetic data. This provides the added advantage that now the generated data can be used for other similar tasks. We provide the theoretical guarantees and empirical results in support of our approach.
CT Image Synthesis Using Weakly Supervised Segmentation and Geometric Inter-Label Relations For COVID Image Analysis

While medical image segmentation is an important task for computer aided diagnosis, the high expertise requirement for pixelwise manual annotations makes it a challenging and time consuming task. Since conventional data augmentations do not fully represent the underlying distribution of the training set, the trained models have varying performance when tested on images captured from different sources. Most prior work on image synthesis for data augmentation ignore the interleaved geometric relationship between different anatomical labels. We propose improvements over previous GAN-based medical image synthesis methods by learning the relationship between different anatomical labels. We use a weakly supervised segmentation method to obtain pixel level semantic label map of images which is used learn the intrinsic relationship of geometry and shape across semantic labels. Latent space variable sampling results in diverse generated images from a base image and improves robustness. We use the synthetic images from our method to train networks for segmenting COVID-19 infected areas from lung CT images. The proposed method outperforms state-of-the-art segmentation methods on a public dataset. Ablation studies also demonstrate benefits of integrating geometry and diversity.
NoiseGrad: enhancing explanations by introducing stochasticity to model weights

Many efforts have been made for revealing the decision-making process of black-box learning machines such as deep neural networks, resulting in useful local and global explanation methods. For local explanation, stochasticity is known to help: a simple method, called SmoothGrad, has improved the visual quality of gradient-based attribution by adding noise to the input space and averaging the explanations of the noisy inputs. In this paper, we extend this idea and propose NoiseGrad that enhances both local and global explanation methods. Specifically, NoiseGrad introduces stochasticity in the weight parameter space, such that the decision boundary is perturbed. NoiseGrad is expected to enhance the local explanation, similarly to SmoothGrad, due to the dual relationship between the input perturbation and the decision boundary perturbation. We evaluate NoiseGrad and its fusion with SmoothGrad - FusionGrad - qualitatively and quantitatively with several evaluation criteria, and show that our novel approach significantly outperforms the baseline methods. Both NoiseGrad and FusionGrad are method-agnostic and as handy as SmoothGrad using a simple heuristic for the choice of the hyperparameter setting without the need of fine-tuning.
Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments

Domain generalization aims at performing well on unseen test environments with data from a limited number of training environments. Despite a proliferation of proposal algorithms for this task, assessing their performance both theoretically and empirically is still very challenging. Distributional matching algorithms such as (Conditional) Domain Adversarial Networks [Ganin et al., 2016, Long et al., 2018] are popular and enjoy empirical success, but they lack formal guarantees. Other approaches such as Invariant Risk Minimization (IRM) require a prohibitively large number of training environments -- linear in the dimension of the spurious feature space $d_s$ -- even on simple data models like the one proposed by [Rosenfeld et al., 2021]. Under a variant of this model, we show that both ERM and IRM cannot generalize with $o(d_s)$ environments. We then present an iterative feature matching algorithm that is guaranteed with high probability to yield a predictor that generalizes after seeing only $O(\log d_s)$ environments. Our results provide the first theoretical justification for a family of distribution-matching algorithms widely used in practice under a concrete nontrivial data model.
Constructing driver Hamiltonians for optimization problems with linear constraints

Recent advances in the field of adiabatic quantum computing and the closely related field of quantum annealing have centered around using more advanced and novel Hamiltonian representations to solve optimization problems. One of these advances has centered around the development of driver Hamiltonians that commute with the constraints of an optimization problem—allowing for another avenue to satisfying those constraints instead of imposing penalty terms for each of them. In particular, the approach is able to use sparser connectivity to embed several practical problems on quantum devices in comparison to the standard approach of using penalty terms. However, designing the driver Hamiltonians that successfully commute with several constraints has largely been based on strong intuition for specific problems and with no simple general algorithm for generating them for arbitrary constraints. In this work, we develop a simple and intuitive algebraic framework for reasoning about the commutation of Hamiltonians with linear constraints—one that allows us to classify the complexity of finding a driver Hamiltonian for an arbitrary set of linear constraints as NP-complete. Because unitary operators are exponentials of Hermitian operators, these results can also be applied to the construction of mixers in the quantum alternating operator ansatz framework.