Miracle-dz/UniFAR-FTUs huggingface dataset:
Survey of Image Based Graph Neural Networks

In this survey paper, we analyze image based graph neural networks and propose a three-step classification approach. We first convert the image into superpixels using the Quickshift algorithm so as to reduce 30% of the input data. The superpixels are subsequently used to generate a region adjacency graph. Finally, the graph is passed through a state-of-art graph convolutional neural network to get classification scores. We also analyze the spatial and spectral convolution filtering techniques in graph neural networks. Spectral-based models perform better than spatial-based models and classical CNN with lesser compute cost.
On accretion discs formed in MHD simulations of black hole–neutron star mergers with accurate microphysics

Remnant accretion discs formed in compact object mergers are an important ingredient in the understanding of electromagnetic afterglows of multimessenger gravitational-wave events. Due to magnetically and neutrino-driven winds, a significant fraction of the disc mass will eventually become unbound and undergo r-process nucleosynthesis. While this process has been studied in some detail, previous studies have typically used approximate initial conditions for the accretion discs, or started from purely hydrodynamical simulations. In this work, we analyse the properties of accretion discs formed from near equal-mass black hole–neutron star mergers simulated in general-relativistic magnetohydrodynamics in dynamical spacetimes with an accurate microphysical description. The post-merger systems were evolved until $120\, {\rm ms}$ for different finite-temperature equations of state and black hole spins. We present a detailed analysis of the fluid properties and of the magnetic-field topology. In particular, we provide analytic fits of the magnetic-field strength and specific entropy as a function of the rest-mass density, which can be used for the construction of equilibrium disc models. Finally, we evolve one of the systems for a total of $350\, \rm ms$ after merger and study the prospect for eventual jet launching. While our simulations do not reach this stage, we find clear evidence of continued funnel magnetization and clearing, a prerequisite for any jet-launching mechanism.
Learning the Precise Feature for Cluster Assignment

Clustering is one of the fundamental tasks in computer vision and pattern recognition. Recently, deep clustering methods (algorithms based on deep learning) have attracted wide attention with their impressive performance. Most of these algorithms combine deep unsupervised representation learning and standard clustering together. However, the separation of representation learning and clustering will lead to suboptimal solutions because the two-stage strategy prevents representation learning from adapting to subsequent tasks (e.g., clustering according to specific cues). To overcome this issue, efforts have been made in the dynamic adaption of representation and cluster assignment, whereas current state-of-the-art methods suffer from heuristically constructed objectives with the representation and cluster assignment alternatively optimized. To further standardize the clustering problem, we audaciously formulate the objective of clustering as finding a precise feature as the cue for cluster assignment. Based on this, we propose a general-purpose deep clustering framework, which radically integrates representation learning and clustering into a single pipeline for the first time. The proposed framework exploits the powerful ability of recently developed generative models for learning intrinsic features, and imposes an entropy minimization on the distribution of the cluster assignment by a dedicated variational algorithm. The experimental results show that the performance of the proposed method is superior, or at least comparable to, the state-of-the-art methods on the handwritten digit recognition, fashion recognition, face recognition, and object recognition benchmark datasets.
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective

Large width limits have been a recent focus of deep learning research: modulo computational practicalities, do wider networks outperform narrower ones? Answering this question has been challenging, as conventional networks gain representational power with width, potentially masking any negative effects. Our analysis in this paper decouples capacity and width via the generalization of neural networks to Deep Gaussian Processes (Deep GP), a class of nonparametric hierarchical models that subsume neural nets. In doing so, we aim to understand how width affects (standard) neural networks once they have sufficient capacity for a given modeling task. Our theoretical and empirical results on Deep GP suggest that large width can be detrimental to hierarchical models. Surprisingly, we prove that even nonparametric Deep GP converge to Gaussian processes, effectively becoming shallower without any increase in representational power. The posterior, which corresponds to a mixture of data-adaptable basis functions, becomes less data-dependent with width. Our tail analysis demonstrates that width and depth have opposite effects: depth accentuates a model's non-Gaussianity, while width makes models increasingly Gaussian. We find there is a"sweet spot"that maximizes test performance before the limiting GP behavior prevents adaptability, occurring at width = 1 or width = 2 for nonparametric Deep GP. These results make strong predictions about the same phenomenon in conventional neural networks trained with L2 regularization (analogous to a Gaussian prior on parameters): we show that such neural networks may need up to 500 - 1000 hidden units for sufficient capacity - depending on the dataset - but further width degrades performance.
Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm

Sparse adversarial attacks can fool deep neural networks (DNNs) by only perturbing a few pixels (regularized by l_0 norm). Recent efforts combine it with another l_infty imperceptible on the perturbation magnitudes. The resultant sparse and imperceptible attacks are practically relevant, and indicate an even higher vulnerability of DNNs that we usually imagined. However, such attacks are more challenging to generate due to the optimization difficulty by coupling the l_0 regularizer and box constraints with a non-convex objective. In this paper, we address this challenge by proposing a homotopy algorithm, to jointly tackle the sparsity and the perturbation bound in one unified framework. Each iteration, the main step of our algorithm is to optimize an l_0-regularized adversarial loss, by leveraging the nonmonotone Accelerated Proximal Gradient Method (nmAPG) for nonconvex programming; it is followed by an l_0 change control step, and an optional post-attack step designed to escape bad local minima. We also extend the algorithm to handling the structural sparsity regularizer. We extensively examine the effectiveness of our proposed homotopy attack for both targeted and non-targeted attack scenarios, on CIFAR-10 and ImageNet datasets. Compared to state-of-the-art methods, our homotopy attack leads to significantly fewer perturbations, e.g., reducing 42.91% on CIFAR-10 and 75.03% on ImageNet (average case, targeted attack), at similar maximal perturbation magnitudes, when still achieving 100% attack success rates. Our codes are available at: https://github.com/VITA-Group/SparseADV_Homotopy.
Distributionally robust tail bounds based on Wasserstein distance and $f$-divergence

In this work, we provide robust bounds on the tail probabilities and the tail index of heavy-tailed distributions in the context of model misspecification. They are defined as the optimal value when computing the worst-case tail behavior over all models within some neighborhood of the reference model. The choice of the discrepancy between the models used to build this neighborhood plays a crucial role in assessing the size of the asymptotic bounds. We evaluate the robust tail behavior in ambiguity sets based on the Wasserstein distance and Csisz\'ar $f$-divergence and obtain explicit expressions for the corresponding asymptotic bounds. In an application to Danish fire insurance claims we compare the difference between these bounds and show the importance of the choice of discrepancy measure.
Scaling Vision with Sparse Mixture of Experts

Sparsely-gated Mixture of Experts networks (MoEs) have demonstrated excellent scalability in Natural Language Processing. In Computer Vision, however, almost all performant networks are"dense", that is, every input is processed by every parameter. We present a Vision MoE (V-MoE), a sparse version of the Vision Transformer, that is scalable and competitive with the largest dense networks. When applied to image recognition, V-MoE matches the performance of state-of-the-art networks, while requiring as little as half of the compute at inference time. Further, we propose an extension to the routing algorithm that can prioritize subsets of each input across the entire batch, leading to adaptive per-image compute. This allows V-MoE to trade-off performance and compute smoothly at test-time. Finally, we demonstrate the potential of V-MoE to scale vision models, and train a 15B parameter model that attains 90.35% on ImageNet.
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Driven Adaptive Prior

Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density. The framework assumes the prior noise as a standard Gaussian distribution, whereas the corresponding data distribution may be more complicated than the standard Gaussian distribution, which potentially introduces inefﬁciency in denoising the prior noise into the data sample because of the discrepancy between the data and the prior. In this paper, we propose PriorGrad to improve the efﬁciency of the conditional diffusion model (for example, a vocoder using a mel-spectrogram as the condition) by applying an adaptive prior derived from the data statistics based on the conditional information. We formulate the training and sampling procedures of PriorGrad and demonstrate the advantages of an adaptive prior through a theoretical analysis. Focusing on the audio domain, we consider the recently proposed diffusion-based audio generative models based on both the spectral and time domains and show that PriorGrad achieves a faster convergence leading to data and parameter efﬁciency and improved quality, and thereby demonstrating the efﬁciency of a data-driven adaptive prior.
Applying Endogenous Learning Models in Energy System Optimization

Conventional energy production based on fossil fuels causes emissions that contribute to global warming. Accurate energy system models are required for a cost-optimal transition to a zero-emission energy system, which is an endeavor that requires a methodical modeling of cost reductions due to technological learning effects. In this review, we summarize common methodologies for modeling technological learning and associated cost reductions via learning curves. This is followed by a literature survey to uncover learning rates for relevant low-carbon technologies required to model future energy systems. The focus is on (i) learning effects in hydrogen production technologies and (ii) the application of endogenous learning in energy system models. Finally, we discuss methodological shortcomings of typical learning curves and possible remedies. One of our main results is an up-to-date overview of learning rates that can be applied in energy system models.
Verifying Quantized Neural Networks using SMT-Based Model Checking

Artificial Neural Networks (ANNs) are being deployed for an increasing number of safety-critical applications, including autonomous cars and medical diagnosis. However, concerns about their reliability have been raised due to their black-box nature and apparent fragility to adversarial attacks. These concerns are amplified when ANNs are deployed on restricted system, which limit the precision of mathematical operations and thus introduce additional quantization errors. Here, we develop and evaluate a novel symbolic verification framework using software model checking (SMC) and satisfiability modulo theories (SMT) to check for vulnerabilities in ANNs. More specifically, we propose several ANN-related optimizations for SMC, including invariant inference via interval analysis, slicing, expression simplifications, and discretization of non-linear activation functions. With this verification framework, we can provide formal guarantees on the safe behavior of ANNs implemented both in floating- and fixed-point arithmetic. In this regard, our verification approach was able to verify and produce adversarial examples for $52$ test cases spanning image classification and general machine learning applications. Furthermore, for small- to medium-sized ANN, our approach completes most of its verification runs in minutes. Moreover, in contrast to most state-of-the-art methods, our approach is not restricted to specific choices regarding activation functions and non-quantized representations. Our experiments show that our approach can analyze larger ANN implementations and substantially reduce the verification time compared to state-of-the-art techniques that use SMT solving.
Numerical Solution of the L1-Optimal Transport Problem on Surfaces

In this article we study the numerical solution of the $L^1$-Optimal Transport Problem on 2D surfaces embedded in $R^3$, via the DMK formulation introduced in [FaccaCardinPutti:2018]. We extend from the Euclidean into the Riemannian setting the DMK model and conjecture the equivalence with the solution Monge-Kantorovich equations, a PDE-based formulation of the $L^1$-Optimal Transport Problem. We generalize the numerical method proposed in [FaccaCardinPutti:2018,FaccaDaneriCardinPutti:2020] to 2D surfaces embedded in $\REAL^3$ using the Surface Finite Element Model approach to approximate the Laplace-Beltrami equation arising from the model. We test the accuracy and efficiency of the proposed numerical scheme, comparing our approximate solution with respect to an exact solution on a 2D sphere. The results show that the numerical scheme is efficient, robust, and more accurate with respect to other numerical schemes presented in the literature for the solution of ls$L^1$-Optimal Transport Problem on 2D surfaces.
Hardness of Approximate Diameter: Now for Undirected Graphs

Approximating the graph diameter is a basic task of both theoretical and practical interest. A simple folklore algorithm can output a 2-approximation to the diameter in linear time by running BFS from an arbitrary vertex. It has been open whether a better approximation is possible in near-linear time. A series of papers on fine-grained complexity have led to strong hardness results for diameter in directed graphs, culminating in a recent tradeoff curve independently discovered by [Li, STOC'21] and [Dalirrooyfard and Wein, STOC'21], showing that under the Strong Exponential Time Hypothesis (SETH), for any integer $k\geq 2$ and $\delta > 0$, a $2-\frac{1}{k}-\delta$ approximation for diameter in directed $m$-edge graphs requires $mn^{1+1/(k-1)-o(1)}$ time. In particular, the simple linear time 2-approximation algorithm is optimal for directed graphs. In this paper we prove that the same tradeoff lower bound curve is possible for undirected graphs as well, extending results of [Roditty and Vassilevska W. , STOC'13], [Li'20] and [Bonnet, ICALP'21] who proved the first few cases of the curve, $k=2,3$ and 4, respectively. Our result shows in particular that the simple linear time 2-approximation algorithm is also optimal for undirected graphs. To obtain our result we develop new tools for fine-grained reductions that could be useful for proving SETH-based hardness for other problems in undirected graphs related to distance computation.
A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling

The Variational Autoencoder (VAE) is a powerful deep generative model that is now extensively used to represent high-dimensional complex data via a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, input data vectors are processed independently. In recent years, a series of papers have presented different extensions of the VAE to process sequential data, that not only model the latent space, but also model the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks. We recently performed a comprehensive review of those models and unified them into a general class called Dynamical Variational Autoencoders (DVAEs). In the present paper, we present the results of an experimental benchmark comparing six of those DVAE models on the speech analysis-resynthesis task, as an illustration of the high potential of DVAEs for speech modeling.
On the separation of Hall and Ohmic nonlinear responses

The symmetric and antisymmetric parts of the linear conductivity describe the dissipative (Ohmic) and nondissipative (Hall) parts of the current. The Hall current is always transverse to the applied electric field regardless of its orientation; the Ohmic current is purely longitudinal in cubic crystals, but in lower-symmetry crystals it has a transverse component whenever the field is not aligned with a principal axis. In this work, we extend that analysis beyond the linear regime. We consider all possible ways of partitioning the current at any order in the electric field without taking symmetry into account, and find that the Hall vs Ohmic decomposition is the only one that satisfies certain basic requirements. A general prescription is given for achieving that decomposition, and the case of the quadratic conductivity is analyzed in detail. By performing a symmetry analysis we find that in five of the 122 magnetic point groups the quadratic dc conductivity is purely Ohmic and even under time reversal, a type of response that is entirely disorder mediated.
Neuroadaptive electroencephalography: a proof-of-principle study in infants

A core goal of functional neuroimaging is to study how the environment is processed in the brain. The mainstream paradigm involves concurrently measuring a broad spectrum of brain responses to a small set of environmental features preselected with reference to previous studies or a theoretical framework. As a complement, we invert this approach by allowing the investigator to record the modulation of a preselected brain response by a broad spectrum of environmental features. Our approach is optimal when theoretical frameworks or previous empirical data are impoverished. By using a prespecified closed-loop design, the approach addresses fundamental challenges of reproducibility and generalisability in brain research. These conditions are particularly acute when studying the developing brain, where our theories based on adult brain function may fundamentally misrepresent the topography of infant cognition and where there are substantial practical challenges to data acquisition. Our methodology employs machine learning to map modulation of a neural feature across a space of experimental stimuli. Our method collects, processes and analyses EEG brain data in real-time; and uses a neuro-adaptive Bayesian optimisation algorithm to adjust the stimulus presented depending on the prior samples of a given participant. Unsampled stimuli can be interpolated by fitting a Gaussian process regression along the dataset. We show that our method can automatically identify the face of the infant's mother through online recording of their Nc brain response to a face continuum. We can retrieve model statistics of individualised responses for each participant, opening the door for early identification of atypical development. This approach has substantial potential in infancy research and beyond for improving power and generalisability of mapping the individual cognitive topography of brain function.
Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

Several recent end-to-end text-to-speech (TTS) models enabling single-stage training and parallel sampling have been proposed, but their sample quality does not match that of two-stage TTS systems. In this work, we present a parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models. Our method adopts variational inference augmented with normalizing flows and an adversarial training process, which improves the expressive power of generative modeling. We also propose a stochastic duration predictor to synthesize speech with diverse rhythms from input text. With the uncertainty modeling over latent variables and the stochastic duration predictor, our method expresses the natural one-to-many relationship in which a text input can be spoken in multiple ways with different pitches and rhythms. A subjective human evaluation (mean opinion score, or MOS) on the LJ Speech, a single speaker dataset, shows that our method outperforms the best publicly available TTS systems and achieves a MOS comparable to ground truth.
Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap

We provide a unifying view of a large family of previous imitation learning algorithms through the lens of moment matching. At its core, our classification scheme is based on whether the learner attempts to match (1) reward or (2) action-value moments of the expert's behavior, with each option leading to differing algorithmic approaches. By considering adversarially chosen divergences between learner and expert behavior, we are able to derive bounds on policy performance that apply for all algorithms in each of these classes, the first to our knowledge. We also introduce the notion of moment recoverability, implicit in many previous analyses of imitation learning, which allows us to cleanly delineate how well each algorithmic family is able to mitigate compounding errors. We derive three novel algorithm templates (AdVIL, AdRIL, and DAeQuIL) with strong guarantees, simple implementation, and competitive empirical performance.
Towards User-Driven Neural Machine Translation

A good translation should not only translate the original content semantically, but also incarnate personal traits of the original text. For a real-world neural machine translation (NMT) system, these user traits (e.g., topic preference, stylistic characteristics and expression habits) can be preserved in user behavior (e.g., historical inputs). However, current NMT systems marginally consider the user behavior due to: 1) the difficulty of modeling user portraits in zero-shot scenarios, and 2) the lack of user-behavior annotated parallel dataset. To fill this gap, we introduce a novel framework called user-driven NMT. Specifically, a cache-based module and a user-driven contrastive learning method are proposed to offer NMT the ability to capture potential user traits from their historical inputs under a zero-shot learning fashion. Furthermore, we contribute the first Chinese-English parallel corpus annotated with user behavior called UDT-Corpus. Experimental results confirm that the proposed user-driven NMT can generate user-specific translations.