Miracle-dz/UniFAR-FTUs huggingface dataset:
Towards Verifying the Geometric Robustness of Large-scale Neural Networks

Deep neural networks (DNNs) are known to be vulnerable to adversarial geometric transformation. This paper aims to verify the robustness of large-scale DNNs against the combination of multiple geometric transformations with a provable guarantee. Given a set of transformations (e.g., rotation, scaling, etc.), we develop GeoRobust, a black-box robustness analyser built upon a novel global optimisation strategy, for locating the worst-case combination of transformations that affect and even alter a network's output. GeoRobust can provide provable guarantees on finding the worst-case combination based on recent advances in Lipschitzian theory. Due to its black-box nature, GeoRobust can be deployed on large-scale DNNs regardless of their architectures, activation functions, and the number of neurons. In practice, GeoRobust can locate the worst-case geometric transformation with high precision for the ResNet50 model on ImageNet in a few seconds on average. We examined 18 ImageNet classifiers, including the ResNet family and vision transformers, and found a positive correlation between the geometric robustness of the networks and the parameter numbers. We also observe that increasing the depth of DNN is more beneficial than increasing its width in terms of improving its geometric robustness. Our tool GeoRobust is available at https://github.com/TrustAI/GeoRobust.
A Comparison of Conditioned Versus Unconditioned Forecasts of the VAR(1) Process

ABSTRACT The properties of a forecast usually depend upon whether or not the forecast is conditioned on the final period observation. In the case of unconditioned forecasts, it is well known that the point predictions are unbiased. If, on the other hand, the forecast is conditional, then the forecast may be biased. Existing analytical results in literature are insufficient for describing the properties of the conditioned forecast properly, particularly in multivariate models. This article examines some finite sample properties of conditioned forecasts of the VAR(1) process by means of Monte Carlo experiments. We use a number of parameter settings for the VAR(1) process to demonstrate that the forecast bias of the conditioned forecast may be considerable. Hence, unless the analyst has a clear idea of whether the conditioned or unconditioned forecast is relevant for the time series being analyzed, statistical inferences may be seriously erratic.
On Heterogeneous Treatment Effects in Heterogeneous Causal Graphs

Heterogeneity and comorbidity are two interwoven challenges associated with various healthcare problems that greatly hampered research on developing effective treatment and understanding of the underlying neurobiological mechanism. Very few studies have been conducted to investigate heterogeneous causal effects (HCEs) in graphical contexts due to the lack of statistical methods. To characterize this heterogeneity, we first conceptualize heterogeneous causal graphs (HCGs) by generalizing the causal graphical model with confounder-based interactions and multiple mediators. Such confounders with an interaction with the treatment are known as moderators. This allows us to flexibly produce HCGs given different moderators and explicitly characterize HCEs from the treatment or potential mediators on the outcome. We establish the theoretical forms of HCEs and derive their properties at the individual level in both linear and nonlinear models. An interactive structural learning is developed to estimate the complex HCGs and HCEs with confidence intervals provided. Our method is empirically justified by extensive simulations and its practical usefulness is illustrated by exploring causality among psychiatric disorders for trauma survivors.
Adversarially Slicing Generative Networks: Discriminator Slices Feature for One-Dimensional Optimal Transport

Generative adversarial networks (GANs) learn a target probability distribution by optimizing a generator and a discriminator with minimax objectives. This paper addresses the question of whether such optimization actually provides the generator with gradients that make its distribution close to the target distribution. We derive sufﬁcient conditions for the discriminator to serve as the distance between the distributions by connecting the GAN formulation with the concept of sliced optimal transport. Furthermore, by leveraging these theoretical results, we propose a novel GAN training scheme, called adversarially slicing generative network (ASGN). With only simple modiﬁcations, the ASGN is applicable to a broad class of existing GANs. Experiments on synthetic and image datasets support our theoretical results and the ASGN’s effectiveness as compared to usual GANs.
Towards Inference Efficient Deep Ensemble Learning

Ensemble methods can deliver surprising performance gains but also bring significantly higher computational costs, e.g., can be up to 2048X in large-scale ensemble tasks. However, we found that the majority of computations in ensemble methods are redundant. For instance, over 77% of samples in CIFAR-100 dataset can be correctly classified with only a single ResNet-18 model, which indicates that only around 23% of the samples need an ensemble of extra models. To this end, we propose an inference efficient ensemble learning method, to simultaneously optimize for effectiveness and efficiency in ensemble learning. More specifically, we regard ensemble of models as a sequential inference process and learn the optimal halting event for inference on a specific sample. At each timestep of the inference process, a common selector judges if the current ensemble has reached ensemble effectiveness and halt further inference, otherwise filters this challenging sample for the subsequent models to conduct more powerful ensemble. Both the base models and common selector are jointly optimized to dynamically adjust ensemble inference for different samples with various hardness, through the novel optimization goals including sequential ensemble boosting and computation saving. The experiments with different backbones on real-world datasets illustrate our method can bring up to 56% inference cost reduction while maintaining comparable performance to full ensemble, achieving significantly better ensemble utility than other baselines. Code and supplemental materials are available at https://seqml.github. io/irene.
Numerical simulations of sound radiated from an axisymmetric premixed reacting jet

The sound radiated by an axisymmetric (two-dimensional) premixed reacting free jet was studied using direct numerical simulation. The jet was injected into cold combustion products. A narrow (in radial extent) high temperature pulse was specified at the jet inlet to stabilize the reacting jet. The computational domain included both the near-field flow and far-field acoustic regions. Both reacting and nonreacting cases were considered. The heat release associated with the reacting jet had a significant effect on the vortical structure, as well as the sound radiation level and pattern, within the jet. The sound radiation pattern and the source terms in Lighthill’s equation were used to identify apparent sound source locations. Within the context of the assumptions of the present simulations, the results showed that the effect of heat release was to: (1) Stabilize the jet, (2) enhance sound radiation levels due to an increase in the entropy source, and (3) shift the frequency of the most unstable mode to low.
Enhancing Neural Network Differential Equation Solvers

We motivate the use of neural networks for the construction of numerical solutions to diﬀerential equations. We prove that there exists a feed-forward neural network that can arbitrarily minimise an objective function that is zero at the solution of Poisson’s equation, allowing us to guarantee that neural network solution estimates can get arbitrarily close to the exact solutions. We also show how these estimates can be appreciably enhanced through various strategies, in particular through the construction of error correction networks, for which we propose a general method. We conclude by providing numerical experiments that attest to the validity of all such strategies for variants of Poisson’s equation. The source code for this project can be found at https://github.com/mjhwright/error-correction .
Curvature Filtrations for Graph Generative Model Evaluation

Graph generative model evaluation necessitates understanding differences between graphs on the distributional level. This entails being able to harness salient attributes of graphs in an efficient manner. Curvature constitutes one such property that has recently proved its utility in characterising graphs. Its expressive properties, stability, and practical utility in model evaluation remain largely unexplored, however. We combine graph curvature descriptors with emerging methods from topological data analysis to obtain robust, expressive descriptors for evaluating graph generative models.
Compression, Generalization and Learning

A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change is interpreted that this observation brings in extra information and, in learning theory, this corresponds to misclassification, or misprediction. In this paper, we lay the foundations of a new theory that allows one to keep control on the probability of change of compression (which maps into the statistical"risk"in learning applications). Under suitable conditions, the cardinality of the compressed set is shown to be a consistent estimator of the probability of change of compression (without any upper limit on the size of the compressed set); moreover, unprecedentedly tight finite-sample bounds to evaluate the probability of change of compression are obtained under a generally applicable condition of preference. All results are usable in a fully agnostic setup, i.e., without requiring any a priori knowledge on the probability distribution of the observations. Not only these results offer a valid support to develop trust in observation-driven methodologies, they also play a fundamental role in learning techniques as a tool for hyper-parameter tuning.
Attribute-Guided Multi-Level Attention Network for Fine-Grained Fashion Retrieval

This paper proposes an attribute-guided multi-level attention network (AG-MLAN) to learn fine-grained fashion similarity. AG-MLAN is able to make a more accurate attribute positioning and capture more discriminative features under the guidance of the specified attribute. Specifically, the AG-MLAN contains two branches, branch 1 aims to force the model to recognize different attributes, while branch 2 aims to learn multiple attribute-specific embedding spaces for measuring the fine-grained similarity. We first improve the Convolutional Neural Network (CNN) backbone to extract hierarchical feature representations, then the extracted feature representations are passed into branch 1 for attribute classification and branch 2 for multi-level feature extraction. In branch 2, we first propose a multi-level attention module to extract a more discriminative representation under the guidance of a specific attribute. Then, we adopt a masked embedding module to learn attribute-aware embedding. Finally, the AG-MLAN is trained with a weighted loss of the classification loss in branch 1 and the triplet loss of the masked embedding features in branch 2 to further improve the accuracy in attribute location. Extensive experiments on the DeepFashion, FashionAI, and Zappos50k datasets show the effectiveness of AG-MLAN for fine-grained fashion similarity learning and its potential for attribute-guided retrieval tasks. The proposed AG-MLAN outperforms the state-of-the-art methods in the fine-grained fashion similarity retrieval task.
T IME OUT OF M IND : G ENERATING E MOTIONALLY C ONDITIONED R ATE OF S PEECH

Voice synthesis has seen signiﬁcant improvements in the past decade resulting in highly intelligible voices. Further investigations have resulted in models that can produce variable speech, including conditional emotional expression. The problem lies, however, in a focus on phrase level modiﬁcations and prosodic vocal features. Using the CREMA-D dataset we have trained a GAN conditioned on emotion to generate worth lengths for a given input text. These word lengths are relative to neutral speech and can be provided, through speech synthesis markup language (SSML) to a text to speech (TTS) system to generate more expressive speech. We were able to achieve better performances on objective measures for neutral speech, and better time alignment for happy speech when compared to an out of box model. However, further investigation of subjective evaluation is required.
StriderNET: A Graph Reinforcement Learning Approach to Optimize Atomic Structures on Rough Energy Landscapes

Optimization of atomic structures presents a challenging problem, due to their highly rough and non-convex energy landscape, with wide applications in the fields of drug design, materials discovery, and mechanics. Here, we present a graph reinforcement learning approach, StriderNET, that learns a policy to displace the atoms towards low energy configurations. We evaluate the performance of StriderNET on three complex atomic systems, namely, binary Lennard-Jones particles, calcium silicate hydrates gel, and disordered silicon. We show that StriderNET outperforms all classical optimization algorithms and enables the discovery of a lower energy minimum. In addition, StriderNET exhibits a higher rate of reaching minima with energies, as confirmed by the average over multiple realizations. Finally, we show that StriderNET exhibits inductivity to unseen system sizes that are an order of magnitude different from the training system.
Simulation of phonon transmission through graphene and graphene nanoribbons with a Green’s function method

In this paper, phonon transmission through a graphene sheet and graphene nanoribbons is investigated using an atomistic Green’s function method. Best-fit results from first-principles calculations using a fourth nearest neighbor force-constant model are used to establish the matrices that describe interactions among carbon atoms. The effect of carbon isotopes on thermal conductance is investigated, and the results reveal that isotopic doping moderately reduces both phonon transmission function and thermal conductance. The phonon transmission function of each vibrational branch in the heterogeneous interface is also calculated, and comparisons indicate the major and minor channels of phonon transport through graphene. Further, phonon wave effects in zigzag and armchair edge ribbons are investigated. Phonon transmission functions and thermal conductances are found to be sensitive to the edge shape of structures. The phonon transmission functions of nanoribbons with defects are evaluated by artificially crea.
Skew group rings and maximal orders

Let S be a prime Noetherian ring and G a finite group acting on 5 such that Gis x-outer on S. We give sufficient conditions for the skew group ring S * Gto be a prime maximal order. If we impose the further hypothesis that the order of Gbe a unit of S, then these conditions are also necessary. Moreover, if S is a commutative Noetherian domain, then there are necessary and sufficient conditions for S*Gto be a prime maximal order, without requiring that the order of G be a unit in S.
Dielectric Properties of Frequency Modulation Twisted Nematic LCDs Doped with Silver Nanoparticles

Twisted nematic liquid crystal display (TN-LCD) devices, composed of nematic liquid crystal (NLC), doped with silver (Ag) nanoparticles, which are protected with the same NLC molecules of 5CB (Merck, K-15), show electrooptic characteristics that are featured by a frequency modulation and fast response. This device is called FM-TN-LCD. In this paper, we report that the frequency range in the FM-TN-LCD spreading from 40 Hz to an upper limit 2 kHz correlates with that of the dielectric dispersion of the sample cell, which is expressed analytically by the Maxwell–Wagner theory on the heterogeneous dielectric medium, where the dielectric relaxation frequency, and thus the dielectric relaxation time, are primarily governed by the electrical conductivity of the doped Ag nanoparticles.
8D oscillator and 5D Kepler problem: The case of nontrivial constraints

The topologically nontrivial correspondence between the 5D SU(2)-Kepler problem and the 8D singular oscillator is considered. It is shown that both “isospinor” and “isovector” particles on the SU(2) instanton + 1/R background in 5D can be described in terms of an 8D singular oscillator. The energy spectrum is calculated for an arbitrary “isospin” case.
Surface waves induced magnetic reconnection and quantification of space weather phenomena ii

The surface wave induced magnetic reconnection model (SWIMR), based on Alfven resonance theory near the neutral point when extended to magnetosonic waves is seen to have not only the features of formation of current sheets and plasmoids, as in the incompressible case, but also gives rise to the features caused by the thermal catastrophic loss of equilibrium for the current sheet. From this point of view, the catastrophic model for substorm is reexamined and it is shown in this paper that the catastrophic model can explain not only the onset and expansion phase of the substorm, but also the recovery phase. The correspondence of the energy input due to resonance absorption to the Akasofu parameter is clearly seen. Further, the dependence of the critical velocity for the onset of K-H instability, the generation, existence and propagation of surface waves on the interplanetary parameters is discussed keeping in view that the SWIMR model depends on the dispersion of surface waves. The SWIMR model therefore can be parameterized to understand some aspects of space weather phenomena.
Zero to Hero: Exploiting Null Effects to Achieve Variance Reduction in Experiments with One-sided Triggering

In online experiments where the intervention is only exposed, or "triggered", for a small subset of the population, it is critical to use variance reduction techniques to estimate treatment effects with sufficient precision to inform business decisions. Trigger-dilute analysis is often used in these situations, and reduce the sampling variance of overall intent-to-treat (ITT) effects by an order of magnitude equal to the inverse of the triggering rate; for example, a triggering rate of 5% corresponds to roughly a 20x reduction in variance. To apply trigger-dilute analysis, one needs to know experimental subjects' triggering counterfactual statuses, i.e., the counterfactual behavior of subjects under both treatment and control conditions. In this paper, we propose an unbiased ITT estimator with reduced variance applicable for experiments where the triggering counterfactual status is only observed in the treatment group. Our method is based on the efficiency augmentation idea of CUPED and draws upon identification frameworks from the principal stratification and instrumental variables literature. The unbiasedness of our estimation approach relies on a testable assumption that the augmentation term used for covariate adjustment equals zero in expectation. Unlike traditional covariate adjustment or principal score modeling approaches, our estimator can incorporate both pre-experiment and in-experiment observations. We demonstrate through both a real-world experiment and simulations that our estimator can remain unbiased and achieve precision improvements as large as if triggering status were fully observed, and in some cases can even outperform trigger-dilute analysis.
On wind–wave interaction phenomena at low Reynolds numbers

Abstract After decades of research efforts, wind–wave interaction mechanisms have been recognized as extremely elusive. The reason is the complex nature of the problem, which combines complex coupling mechanisms between turbulent wind and water waves with the presence of multiple governing parameters, such as the friction Reynolds number of the wind, the water depth and the wind fetch. As shown unequivocally here, the use of suitable flow settings allows us to reduce the complex problem of wind–wave interaction to its essential features, mainly as a function of the sole friction Reynolds number of the wind. The resulting numerical solution allows us to study the interactions between water and air layers with their own fluid properties, and to unveil very interesting features, such as an oblique wave pattern travelling upstream and a wave-induced Stokes sublayer. The latter is responsible for a drag reduction mechanism in the turbulent wind. Despite the simulated flow conditions being far from the intense events occurring at the ocean–atmosphere interface, the basic flow phenomena unveiled here may explain some experimental evidence in wind–wave problems. Among other things, the wave-induced Stokes sublayer may shed light on the large scatter of the drag coefficient data in field measurements where swell waves of arbitrary directions are often present. Hence the present results and the developed approach pave the way for the understanding and modelling of the surface fluxes at the ocean–atmosphere interface, which are of overwhelming importance for climate science.
Memoir on the Probability of the Causes of Events

Translated from the original French by S. M. Stigler, University of Chicago. Originally published as "Memoire sur la probabilite des causes par les evenemens," par M. de la Place, Professeur a l'Ecole royal Militaire, in Memoires de Mathematique et de Physique, Presentes a l'Academie Royale des Sciences, par divers Savans & lus dans ses Assembles, Tome Sixieme (1774) 621-656. Reprinted in Laplace's Oeuvres compl&tes 8 27-65.
SMAPPIC: Scalable Multi-FPGA Architecture Prototype Platform in the Cloud

Traditionally, architecture prototypes are built on top of FPGA infrastructure, with two associated problems. First, very large FPGAs are prohibitively expensive for most people and institutions. Second, the burden of FPGA development adds to an already uneasy life of researchers, especially those who focus on software. Large designs that do not fit into a single FPGA exacerbate these issues even more. This work presents SMAPPIC — the first open-source prototype platform for shared memory multi-die architectures on cloud FPGAs. SMAPPIC leverages the OpenPiton/BYOC infrastructure and AWS F1 instances to make FPGA-based prototypes of System-on-Chips, processor cores, accelerators, cache subsystems, etc., cheap, scalable, and straightforward. SMAPPIC enables many use cases that are not possible or significantly more complicated in existing software and FPGA tools. This work has the potential to accelerate the rate of innovation in computer engineering fields in the nearest future.
Stepwise Debugging for Hardware Accelerators

High-level programming models for hardware design let domain experts quickly produce specialized accelerators. However, tools for debugging these accelerators remain tied to low-level hardware description languages (HDLs). High-level descriptions contain control-flow information that is lost in HDL code. We describe Cider, a stepwise debugger that exploits this information to provide software-like debugging abstractions for languages that compile to hardware. Cider uses Calyx, an intermediate language for accelerator generators that preserves control information. Cider provides breakpoints, watchpoints, state inspection, and source-level position mapping. Using case studies that examine one new and two preexisting accelerator generators, we demonstrate how Cider helps find and localize previously unreported bugs. By directly simulating a control-rich representation, Cider avoids wasting effort on inactive parts of the design and, despite being largely unoptimized, performs competitively with open-source HDL simulators.