Miracle-dz/UniFAR-FTUs huggingface dataset:
Verification-Aided Deep Ensemble Selection

Deep neural networks (DNNs) have become the technology of choice for realizing a variety of complex tasks. However, as highlighted by many recent studies, even an imperceptible perturbation to a correctly classified input can lead to misclassification by a DNN. This renders DNNs vulnerable to strategic input manipulations by attackers, and also over-sensitive to environmental noise. To mitigate this phenomenon, practitioners apply joint classification by an ensemble of DNNs. By aggregating the classification outputs of different individual DNNs for the same input, ensemble-based classification reduces the risk of misclassifications due to the specific realization of the stochastic training process of any single DNN. However, the effectiveness of a DNN ensemble is highly dependent on its members not simultaneously erring on many different inputs. In this case study, we harness recent advances in DNN verification to devise a methodology for identifying ensemble compositions that are less prone to simultaneous errors, even when the input is adversarially perturbed - resulting in more robustly-accurate ensemble-based classification. Our proposed framework uses a DNN verifier as a backend, and includes heuristics that help reduce the high complexity of directly verifying ensembles. More broadly, our work puts forth a novel universal objective for formal verification that can potentially improve the robustness of real-world, deep-learning-based systems across a variety of application domains.
Nesterov Accelerated Shuffling Gradient Method for Convex Optimization

In this paper, we propose Nesterov Accelerated Shuffling Gradient (NASG), a new algorithm for the convex finite-sum minimization problems. Our method integrates the traditional Nesterov's acceleration momentum with different shuffling sampling schemes. We show that our algorithm has an improved rate of $\mathcal{O}(1/T)$ using unified shuffling schemes, where $T$ is the number of epochs. This rate is better than that of any other shuffling gradient methods in convex regime. Our convergence analysis does not require an assumption on bounded domain or a bounded gradient condition. For randomized shuffling schemes, we improve the convergence bound further. When employing some initial condition, we show that our method converges faster near the small neighborhood of the solution. Numerical simulations demonstrate the efficiency of our algorithm.
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers

Generating images from textual descriptions has gained a lot of attention. Recently, DALL-E [44], a multimodal transformer language model, and its variants have shown high-quality text-to-image generation capabilities with a simple architecture and training objective, powered by large-scale training data and computation. However, despite the interesting image generation results, there has not been a detailed analysis on how to evaluate such models. In this work, we investigate the reasoning capabilities and social biases of such text-to-image generative transformers in detail. First, we measure four visual reasoning skills: object recognition, object counting, color recognition, and spatial relation understanding. For this, we propose P AINT S KILLS , a diagnostic dataset and evaluation toolkit that measures these four visual reasoning skills. Second, we measure the text alignment and quality of the generated images based on pretrained image cap-tioning, image-text retrieval, and image classiﬁcation models. Third, we assess social biases in the models. For this, we suggest evaluation of gender and racial biases of text-to-image generation models based on a pretrained image-text retrieval model and human evaluation. In our experiments, we show that recent text-to-image generative trans-former models perform better in recognizing and counting objects than recognizing colors and understanding spatial relations, while there exists a large gap between the model performances and upper bound accuracy on all skills. Next, we demonstrate that recent text-to-image models learn spe-ciﬁc gender/racial biases from web image-text pairs. We also show that our automatic evaluations of visual reasoning skills and gender bias are highly correlated with human judgments. We hope our work will help guide and care-fully measure future research progress in improving text-to-image generation models on challenging
A Tale of Two Datasets: Representativeness and Generalisability of Inference for Samples of Networks

Abstract The last two decades have seen considerable progress in foundational aspects of statistical network analysis, but the path from theory to application is not straightforward. Two large, heterogeneous samples of small networks of within-household contacts in Belgium were collected using two different but complementary sampling designs: one smaller but with all contacts in each household observed, the other larger and more representative but recording contacts of only one person per household. We wish to combine their strengths to learn the social forces that shape household contact formation and facilitate simulation for prediction of disease spread, while generalising to the population of households in the region. To accomplish this, we describe a flexible framework for specifying multi-network models in the exponential family class and identify the requirements for inference and prediction under this framework to be consistent, identifiable, and generalisable, even when data are incomplete; explore how these requirements may be violated in practice; and develop a suite of quantitative and graphical diagnostics for detecting violations and suggesting improvements to candidate models. We report on the effects of network size, geography, and household roles on household contact patterns (activity, heterogeneity in activity, and triadic closure). Supplementary materials for this article are available online.
Electromagnetic response of cuprate superconductors with coexisting electronic nematicity

ABSTRACT The electronically nematic order has emerged as a key feature of cuprate superconductors, however, its correlation with the fundamental properties such as the electromagnetic response remains unclear. Here the nematic-order state strength dependence of the electromagnetic response in cuprate superconductors is investigated within the framework of the kinetic-energy-driven superconductivity. It is shown that a significant anisotropy of the electromagnetic response is caused by the electronic nematicity. In particular, in addition to the pure d-wave component of the superconducting gap, the pure s-wave component of the superconducting gap is generated by the electronic nematicity, therefore there is a coexistence and competition of the d-wave component and the s-wave component. This coexistence and competition leads to that the maximal condensation energy appears at around the optimal strength of the electronic nematicity, and then decreases in both the weak and strong strength regions, which in turn induces the enhancement of superconductivity, and gives rise to the dome-like shape of the nematic-order state strength dependence of the superfluid density.
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

While the general idea of self-supervised learning is identical across modalities, the actual algorithms and objectives differ widely because they were developed with a single modality in mind. To get us closer to general self-supervised learning, we present data2vec, a framework that uses the same learning method for either speech, NLP or computer vision. The core idea is to predict latent representations of the full input data based on a masked view of the input in a self-distillation setup using a standard Transformer architecture. Instead of predicting modality-specific targets such as words, visual tokens or units of human speech which are local in nature, data2vec predicts contextualized latent representations that contain information from the entire input. Experiments on the major benchmarks of speech recognition, image classification, and natural language understanding demonstrate a new state of the art or competitive performance to predominant approaches.
US Salary History Bans -- Strategic Disclosure by Job Applicants and the Gender Pay Gap

I study the effects of US salary history bans which restrict employers from inquiring about job applicants’ pay history during the hiring process, but allow candidates to voluntarily share information. Using a difference-in-differences design, I show that these policies narrowed the gender pay gap significantly by 2 p.p., driven almost entirely by an increase in female earnings. The bans were also successful in weakening the auto-correlation between current and future earnings, especially among job-changers. I provide novel evidence showing that when employers could no longer nudge candidates for information, the likelihood of voluntarily disclosing salary history decreased among job applicants and by 2 p.p. more among women. I then develop a salary negotiation model with asymmetric information, where I allow job applicants to choose whether to reveal pay history, and use this framework to explain my empirical findings on disclosure behavior and gender pay gap. Journal Classifications: J08, J31, J38, J78
Rate-Distortion Theory for Strategic Semantic Communication

This paper analyzes the fundamental limit of the strategic semantic communication problem in which a transmitter obtains a limited number of indirect observations of an intrinsic semantic information source and can then influence the receiver’s decoding by sending a limited number of messages over an imperfect channel. The transmitter and the receiver can have different distortion measures and can make rational decisions about their encoding and decoding strategies, respectively. The decoder can also have some side information (e.g., background knowledge and/or information obtained from previous communications) about the semantic source to assist its interpretation of the semantic information. We focus particularly on the case that the transmitter can commit to an encoding strategy and study the impact of the strategic decision making on the rate distortion of semantic communication. Three equilibrium solution concepts including the optimal Stackelberg equilibrium, robust Stackelberg equilibrium, as well as Nash equilibrium are studied and compared. The optimal encoding and decoding strategy profiles under various equilibrium solutions are derived. We prove that committing to an encoding strategy cannot always bring benefit to the encoder. We provide a feasible condition under which committing to an encoding strategy can always reduce the distortion of semantic communication. We consider an example with a dictionary-based semantic information source to verify our observation.
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably

We investigate the role of noise in optimization algorithms for learning over-parameterized models. Specifically, we consider the recovery of a rank one matrix $Y^*\in R^{d\times d}$ from a noisy observation $Y$ using an over-parameterization model. We parameterize the rank one matrix $Y^*$ by $XX^\top$, where $X\in R^{d\times d}$. We then show that under mild conditions, the estimator, obtained by the randomly perturbed gradient descent algorithm using the square loss function, attains a mean square error of $O(\sigma^2/d)$, where $\sigma^2$ is the variance of the observational noise. In contrast, the estimator obtained by gradient descent without random perturbation only attains a mean square error of $O(\sigma^2)$. Our result partially justifies the implicit regularization effect of noise when learning over-parameterized models, and provides new understanding of training over-parameterized neural networks.
Improved estimator for numerical renormalization group calculations of the self-energy

We present a new estimator for the self-energy based on a combination of two equations of motion and discuss its beneﬁts for numerical renormalization group (NRG) calculations. In challenging regimes, NRG results from the standard estimator, a ratio of two correlators, often suffer from artifacts: The imaginary part of the retarded self-energy is not properly normalized and, at low energies, overshoots to unphysical values and displays wiggles. We show that the new estimator resolves the artifacts in these properties as they can be determined directly from the imaginary parts of auxiliary correlators and do not involve real parts obtained by Kramers–Kronig transform. Furthermore, we ﬁnd that the new estimator yields converged results with reduced numerical effort (for a lower number of kept states) and thus is highly valuable when applying NRG to multiorbital systems. Our analysis is targeted at NRG treatments of quantum impurity models, especially those arising within dynamical mean-ﬁeld theory, but most results can be straightforwardly generalized to other impurity or cluster solvers.
Online error control for platform trials

Platform trials evaluate multiple experimental treatments under a single master protocol, where new treatment arms are added to the trial over time. Given the multiple treatment comparisons, there is the potential for inﬂation of the overall type I error rate, which is complicated by the fact that the hypotheses are tested at diﬀerent times and are not all necessarily pre-speciﬁed. Online error control methodology provides a possible solution to the problem of multiplicity for platform trials where a relatively large number of hypotheses are expected to be tested over time. In the online testing framework, hypotheses are tested in a sequential manner, where at each time-step an analyst decides whether to reject the current null hypothesis without knowledge of future tests but based solely on past decisions. Methodology has recently been developed for online control of the false discovery rate as well as the familywise error rate (FWER). In this paper, we describe how to apply online error control to the platform trial setting, present extensive simulation results, and give some recommendations for the use of this new methodology in practice. We show that the algorithms for online error rate control can have a substantially lower FWER than uncorrected testing, while still achieving noticeable gains in power when compared with the use of a Bonferroni procedure. We also illustrate how online error control would have impacted a currently ongoing platform trial.
Multiple Removal: Open Issues, Pressing Challenges and Recent Progress Towards Providing the Next and Higher Level of Required Capability

The demand for new and improved capability in removing multiples is driven by the portfolio of the petroleum industry and by current and anticipated future exploration trends. For example, the industry moved to deep water roughly 30 years ago. With that move, highly effective multiple-removal methods that were being applied industry-wide suddenly bumped up against their statistical assumptions, when applied to deep water plays, and failed.
What's Cracking? A Review and Analysis of Deep Learning Methods for Structural Crack Segmentation, Detection and Quantification

Surface cracks are a very common indicator of potential structural faults. Their early detection and monitoring is an important factor in structural health monitoring. Left untreated, they can grow in size over time and require expensive repairs or maintenance. With recent advances in computer vision and deep learning algorithms, the automatic detection and segmentation of cracks for this monitoring process have become a major topic of interest. This review aims to give researchers an overview of the published work within the field of crack analysis algorithms that make use of deep learning. It outlines the various tasks that are solved through applying computer vision algorithms to surface cracks in a structural health monitoring setting and also provides in-depth reviews of recent fully, semi and unsupervised approaches that perform crack classification, detection, segmentation and quantification. Additionally, this review also highlights popular datasets used for cracks and the metrics that are used to evaluate the performance of those algorithms. Finally, potential research gaps are outlined and further research directions are provided.
Calibrated Learning to Defer with One-vs-All Classifiers

The learning to defer (L2D) framework has the potential to make AI systems safer. For a given input, the system can defer the decision to a human if the human is more likely than the model to take the correct action. We study the calibration of L2D systems, investigating if the probabilities they output are sound. We find that Mozannar&Sontag's (2020) multiclass framework is not calibrated with respect to expert correctness. Moreover, it is not even guaranteed to produce valid probabilities due to its parameterization being degenerate for this purpose. We propose an L2D system based on one-vs-all classifiers that is able to produce calibrated probabilities of expert correctness. Furthermore, our loss function is also a consistent surrogate for multiclass L2D, like Mozannar&Sontag's (2020). Our experiments verify that not only is our system calibrated, but this benefit comes at no cost to accuracy. Our model's accuracy is always comparable (and often superior) to Mozannar&Sontag's (2020) model's in tasks ranging from hate speech detection to galaxy classification to diagnosis of skin lesions.
A Novel Plug-in Module for Fine-Grained Visual Classification

Visual classification can be divided into coarse-grained and fine-grained classification. Coarse-grained classification represents categories with a large degree of dissimilarity, such as the classification of cats and dogs, while fine-grained classification represents classifications with a large degree of similarity, such as cat species, bird species, and the makes or models of vehicles. Unlike coarse-grained visual classification, fine-grained visual classification often requires professional experts to label data, which makes data more expensive. To meet this challenge, many approaches propose to automatically find the most discriminative regions and use local features to provide more precise features. These approaches only require image-level annotations, thereby reducing the cost of annotation. However, most of these methods require two- or multi-stage architectures and cannot be trained end-to-end. Therefore, we propose a novel plug-in module that can be integrated to many common backbones, including CNN-based or Transformer-based networks to provide strongly discriminative regions. The plugin module can output pixel-level feature maps and fuse filtered features to enhance fine-grained visual classification. Experimental results show that the proposed plugin module outperforms state-of-the-art approaches and significantly improves the accuracy to 92.77\% and 92.83\% on CUB200-2011 and NABirds, respectively. We have released our source code in Github https://github.com/chou141253/FGVC-PIM.git.
GLPU: A Geometric Approach For Lidar Pointcloud Upsampling

In autonomous driving, lidar is inherent for the understanding of the 3D environment. Lidar sensors vary in vertical resolutions, where a denser pointcloud depicts a more detailed environment, albeit at a signiﬁcantly higher cost. Point-cloud upsampling predicts high-resolution pointclouds from sparser ones to bridge this performance gap at a lower cost. Although many upsampling frameworks have achieved a robust performance, a fair comparison is difﬁcult as they were tested on different datasets and metrics. In this work, we ﬁrst conduct a consistent comparative study to benchmark the existing algorithms on the KITTI dataset. Then, we observe that there are three common factors that hinder the performance: an inefﬁcient data representation, a small receptive ﬁeld, and low-frequency losses. By leveraging the scene geometry, a new self-supervised geometric lidar pointcloud upsampling (GLPU) framework is proposed to address the aforementioned limitations. Our experiments demonstrate the effectiveness and superior performance of GLPU compared to other techniques on the KITTI benchmark.
Fluid inertia and the scallop theorem

In Stokes flow, Purcell’s scallop theorem forbids objects with time-reversible (reciprocal) swimming strokes from moving. In the presence of inertia, this restriction is eased and reciprocally deforming bodies can swim. Recent works that investigated dimer models with finite inertia show interesting results (e.g. switches in swim direction as a function of inertia) but the results vary and seem to be case-specific. Here, we introduce a general model and investigate the behaviour of a simple swimmer, an asymmetric spherical dimer of oscillating length, at small amplitudes at intermediate Reynolds numbers. In our analysis we make the important distinction between particle and fluid inertia, both of which need to be considered separately. We asymptotically expand the Navier-Stokes equations in the small amplitude limit to obtain a system of linear PDEs. Using a combination of numerical (Finite Element) and analytical (reciprocal theorem, method of reflections) methods we solve the system to obtain the dimer’s swim speed and show that there are two mechanisms that give rise to motion: due to boundary conditions (effective slip velocity) and due to Reynolds stresses. Each mechanism is driven by two classes of sphere–sphere interactions, between one sphere’s motion and 1) the oscillating background flow induced by the other’s motion, and 2) a geometrical asymmetry induced by the other’s presence. We can thus unify and explain behaviours observed in other works. Our results show how sensitive, counter-intuitive and rich motility is in the parameter space of finite inertia of particles and fluid.
Robust Hybrid Learning With Expert Augmentation

Hybrid modelling reduces the misspecification of expert models by combining them with machine learning (ML) components learned from data. Similarly to many ML algorithms, hybrid model performance guarantees are limited to the training distribution. Leveraging the insight that the expert model is usually valid even outside the training domain, we overcome this limitation by introducing a hybrid data augmentation strategy termed \textit{expert augmentation}. Based on a probabilistic formalization of hybrid modelling, we demonstrate that expert augmentation, which can be incorporated into existing hybrid systems, improves generalization. We empirically validate the expert augmentation on three controlled experiments modelling dynamical systems with ordinary and partial differential equations. Finally, we assess the potential real-world applicability of expert augmentation on a dataset of a real double pendulum.
Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learning

Kernel-based models such as kernel ridge regression and Gaussian processes are ubiquitous in machine learning applications for regression and optimization. It is well known that a major downside for kernel-based models is the high computational cost; given a dataset of $n$ samples, the cost grows as $\mathcal{O}(n^3)$. Existing sparse approximation methods can yield a significant reduction in the computational cost, effectively reducing the actual cost down to as low as $\mathcal{O}(n)$ in certain cases. Despite this remarkable empirical success, significant gaps remain in the existing results for the analytical bounds on the error due to approximation. In this work, we provide novel confidence intervals for the Nystr\"om method and the sparse variational Gaussian process approximation method, which we establish using novel interpretations of the approximate (surrogate) posterior variance of the models. Our confidence intervals lead to improved performance bounds in both regression and optimization problems.