Miracle-dz/UniFAR-FTUs huggingface dataset:
Memory Defense: More Robust Classification via a Memory-Masking Autoencoder

Many deep neural networks are susceptible to minute perturbations of images that have been carefully crafted to cause misclassification. Ideally, a robust classifier would be immune to small variations in input images, and a number of defensive approaches have been created as a result. One method would be to discern a latent representation which could ignore small changes to the input. However, typical autoencoders easily mingle inter-class latent representations when there are strong similarities between classes, making it harder for a decoder to accurately project the image back to the original high-dimensional space. We propose a novel framework, Memory Defense, an augmented classifier with a memory-masking autoencoder to counter this challenge. By masking other classes, the autoencoder learns class-specific independent latent representations. We test the model's robustness against four widely used attacks. Experiments on the Fashion-MNIST&CIFAR-10 datasets demonstrate the superiority of our model. We make available our source code at GitHub repository: https://github.com/eashanadhikarla/MemDefense
Joint Pilot Optimization, Target Detection and Channel Estimation for Integrated Sensing and Communication Systems

Radar sensing will be integrated into the 6G communication system to support various applications. In this integrated sensing and communication system, a radar target may also be a communication channel scatterer. In this case, the radar and communication channels exhibit certain joint burst sparsity. We propose a two-stage joint pilot optimization, target detection and channel estimation scheme to exploit such joint burst sparsity and pilot beamforming gain to enhance detection/estimation performance. In Stage 1, the base station (BS) sends downlink pilots (DP) for initial target search, and the user sends uplink pilots (UP) for channel estimation. Then the BS performs joint target detection and channel estimation. In Stage 2, the BS exploits the prior information obtained in Stage 1 to optimize the DP signal to further refine the performance. A Turbo Sparse Bayesian inference algorithm is proposed for joint target detection and channel estimation in both stages. The pilot optimization problem in Stage 2 is a semi-definite programming with rank-1 constraints. By replacing the rank-1 constraint with a tight and smooth approximation, we propose an efficient pilot optimization algorithm based on the majorization-minimization (MM) method. Simulations verify the advantages of the proposed scheme.
On Neural Differential Equations

The conjoining of dynamical systems and deep learning has become a topic of great interest. In particular, neural differential equations (NDEs) demonstrate that neural networks and differential equation are two sides of the same coin. Traditional parameterised differential equations are a special case. Many popular neural network architectures, such as residual networks and recurrent networks, are discretisations. NDEs are suitable for tackling generative problems, dynamical systems, and time series (particularly in physics, finance, .) and are thus of interest to both modern machine learning and traditional mathematical modelling. NDEs offer high-capacity function approximation, strong priors on model space, the ability to handle irregular data, memory efficiency, and a wealth of available theory on both sides. This doctoral thesis provides an in-depth survey of the field. Topics include: neural ordinary differential equations (e.g. for hybrid neural/mechanistic modelling of physical systems); neural controlled differential equations (e.g. for learning functions of irregular time series); and neural stochastic differential equations (e.g. to produce generative models capable of representing complex stochastic dynamics, or sampling from complex high-dimensional distributions). Further topics include: numerical methods for NDEs (e.g. reversible differential equations solvers, backpropagation through differential equations, Brownian reconstruction); symbolic regression for dynamical systems (e.g. via regularised evolution); and deep implicit models (e.g. deep equilibrium models, differentiable optimisation). We anticipate this thesis will be of interest to anyone interested in the marriage of deep learning with dynamical systems, and hope it will provide a useful reference for the current state of the art.
The Marginal Labor Supply Disincentives of Welfare: Evidence from Administrative Barriers to Participation

Existing research on the static effects of the manipulation of welfare program benefit parameters on labor supply has allowed only restrictive forms of heterogeneity in preferences. Yet preference heterogeneity implies that the marginal effects on labor supply of welfare expansions and contractions may differ in different time periods with different populations and which sweep out different portions of the distribution of preferences. A new examination of the heavily studied AFDC program uses variation in state-level administrative barriers to entering the program in the late 1980s and early 1990s to estimate the marginal labor supply effects of changes in program participation induced by that variation. The estimates are obtained from a theory-consistent reduced form model which allows for a nonparametric specification of how changes in welfare program participation affect labor supply on the margin. Estimates using a form of local instrumental variables show that the marginal treatment effects are quadratic, rising and then falling as participation rates rise (i.e., becoming more negative then less negative on hours of work). The average work disincentive is not large but that masks some margins where effects are close to zero and some which are sizable. Traditional IV which estimates a weighted average of marginal effects gives a misleading picture of marginal responses. A counterfactual exercise which applies the estimates to three historical reform periods in 1967, 1981, and 1996 when the program tax rate was significantly altered shows that marginal labor supply responses differed in each period because of differences in the level of participation in the period and the composition of who was on the program.
Lightweight Compositional Embeddings for Incremental Streaming Recommendation

Most work in graph-based recommender systems considers a {\em static} setting where all information about test nodes (i.e., users and items) is available upfront at training time. However, this static setting makes little sense for many real-world applications where data comes in continuously as a stream of new edges and nodes, and one has to update model predictions incrementally to reflect the latest state. To fully capitalize on the newly available data in the stream, recent graph-based recommendation models would need to be repeatedly retrained, which is infeasible in practice. In this paper, we study the graph-based streaming recommendation setting and propose a compositional recommendation model -- Lightweight Compositional Embedding (LCE) -- that supports incremental updates under low computational cost. Instead of learning explicit embeddings for the full set of nodes, LCE learns explicit embeddings for only a subset of nodes and represents the other nodes {\em implicitly}, through a composition function based on their interactions in the graph. This provides an effective, yet efficient, means to leverage streaming graph data when one node type (e.g., items) is more amenable to static representation. We conduct an extensive empirical study to compare LCE to a set of competitive baselines on three large-scale user-item recommendation datasets with interactions under a streaming setting. The results demonstrate the superior performance of LCE, showing that it achieves nearly skyline performance with significantly fewer parameters than alternative graph-based models.
QUARK: A Framework for Quantum Computing Application Benchmarking

Quantum computing (QC) is anticipated to provide a speedup over classical approaches for specific problems in optimization, simulation, and machine learning. With the advances in quantum computing toward practical applications, the need to analyze and compare different quantum solutions is increasing. While different low-level benchmarks exist, they often do not provide sufficient insights into real-world application-level performance. We propose an application-centric benchmark method and the QUantum computing Application benchmaRK (QUARK) framework to foster the investigation and creation of application benchmarks for QC. This paper establishes three significant contributions: (1) it makes a case for application-level benchmarks and provides an in-depth "pen and paper" benchmark formulation of two reference problems: robot path and vehicle option optimization from the industrial domain; (2) it proposes the open-source QUARK framework for designing, implementing, executing, and analyzing benchmarks; (3) it provides multiple reference implementations for these two reference problems based on different known, and where needed, extended, classical and quantum algorithmic approaches and analyzes their performance on different types of infrastructures.
Is A2261 a Fossil Galaxy Cluster in a Transitional Dynamical State?

Fossil cluster A2261 is well studied, but previous studies give contradictory results on its dynamical states, such as its X-ray central entropy and magnitude gap. To improve our understanding of its dynamical state, we conduct multiobject spectroscopic observations with Hectospec on the MMT, covering an area out to 5 virial radii from the cluster center, and get improved completeness and membership. Using these new data, we calculate multiple dynamical indicators, including Gaussianity, distance offset, and velocity offset. These indicators suggest that A2261 is moderately relaxed. However, a Dressler–Shectman test reveals a group candidate to the south at a projected distance that is near the virial radius and overlaps with an X-ray tail-like feature. One of the galaxies associated with that group would be sufficiently bright to reduce the fossil magnitude gap. This raises the possibility that A2261 could have recently transited in fossil status, if the group had previously crossed the cluster and is only now found outside. In the cluster outskirts, we see an extended feature of galaxies located on the opposite side of the cluster from the group candidate. On even larger scales, we find that this feature connects, both on the sky and in velocity space, with a long (4.4 Mpc) filamentary structure in the Sloan Digital Sky Survey data. This could support the idea that a group was fed into the cluster through the filament, temporarily breaking the fossil status and resulting in a minor merger that weakly disturbed the intracluster medium of the cluster.
Testing gravity with black hole shadow subrings

The black hole shadow, first observed by the Event Horizon Telescope in 2017, is the newest method for studying black holes and understanding gravity. Much work has gone into understanding the shadow of a Kerr black hole, including all of the complex astrophysics of the accretion disk, and there are numerous studies of the ideal shadow in non-Kerr black holes and exotic compact objects. This paper presents one of the first studies of the black hole shadow of non-Kerr black holes when the illumination source is an accretion disk. In particular, the ability of current and future very long baseline interferometers to estimate the physical parameters of the black hole spacetime and accretion disk is investigated using two different parametrized black hole metrics that encode a number of possible deviations from Kerr. Both the full shadow image and the individual subrings of the shadow are analyzed as the higher order subrings are weakly dependent on the disk physics and may be a more viable observable for studying the spacetime. The results suggest that with current telescope capabilities and any future earth-based telescopes it will be quite difficult to place strong constraints on departures from the Kerr spacetime, primarily due to the low resolution and strong degeneracies between the spacetime parameters. More optimistically, space-based interferometers may be capable of testing the Kerr nature of black holes and general relativity to comparable or better precision than is currently possible with other observations.
Sensitivity Analysis in the Generalization of Experimental Results

Randomized controlled trials (RCT’s) allow researchers to estimate causal eﬀects in an experimental sample with minimal identifying assumptions. However, to generalize or transport a causal eﬀect from an RCT to a target population, researchers must adjust for a set of treatment eﬀect moderators. In practice, it is impossible to know whether the set of moderators has been properly accounted for. In the following paper, I propose a three parameter sensitivity analysis for generalizing or transporting experimental results using weighted estimators, with several advantages over existing methods. First, the framework does not require assumptions on the underlying data generating process for either the experimental sample selection mechanism or treatment eﬀect heterogeneity. Second, I show that the sensitivity parameters are guaranteed to be bounded and propose several tools researchers can use to perform sensitivity analysis: (1) graphical and numerical summaries for researchers to assess how robust a point estimate is to killer confounders; (2) an extreme scenario analysis; and (3) a formal benchmarking approach for researchers to estimate potential sensitivity parameter values using existing data. Finally, I demonstrate that the proposed framework can be easily extended to the class of doubly robust, augmented weighted estimators. The sensitivity analysis framework is applied to a set of Jobs Training Program experiments.
First-passage Brownian functionals with stochastic resetting

We study the statistical properties of first-passage time functionals of a one dimensional Brownian motion in the presence of stochastic resetting. A first-passage functional is defined as V=∫0tfZ[x(τ)] where t f is the first-passage time of a reset Brownian process x(τ), i.e., the first time the process crosses zero. In here, the particle is reset to x R > 0 at a constant rate r starting from x 0 > 0 and we focus on the following functionals: (i) local time Tloc=∫0tfdτδ(x−xR) , (ii) residence time Tres=∫0tfdτθ(x−xR) , and (iii) functionals of the form An=∫0tfdτ[x(τ)]n with n > −2. For first two functionals, we analytically derive the exact expressions for the moments and distributions. Interestingly, the residence time moments reach minima at some optimal resetting rates. A similar phenomena is also observed for the moments of the functional A n . Finally, we show that the distribution of A n for large A n decays exponentially as ∼exp−An/an for all values of n and the corresponding decay length a n is also estimated. In particular, exact distribution for the first passage time under resetting (which corresponds to the n = 0 case) is derived and shown to be exponential at large time limit in accordance with the generic observation. This behavioural drift from the underlying process can be understood as a ramification due to the resetting mechanism which curtails the undesired long Brownian first passage trajectories and leads to an accelerated completion. We confirm our results to high precision by numerical simulations.
Psycho-physiological responses to plant variegation as measured through eye movement, self-reported emotion and cerebral activity

The aim of the study was to investigate the effects of different variegation of interior plant foliage on visual attention, subjective emotions and cerebral activity. Thirty Egyptian male subjects were presented with five different varieties of Hedera helix for 60 s in random orders. During this time, eye movements and cerebral activity were recorded, and subjective data were then gathered via a semantic differential scale. The results show clear evidence in support of the use of green-yellow and bright green-coloured plants in locations in which comfort and calmness are required, while red and dark green plants can be used to improve energy in places such as offices and children’s areas. In contrast, the green-white plants stimulated mostly negative emotions. These findings suggest that landscape and interior designers should pay more attention for choosing foliage colours during indoor decoration.
Algorithms that get old : the case of generative algorithms

Generative IA networks, like the Variational Auto-Encoders (VAE), and Generative Adversarial Networks (GANs) produce new objects each time when asked to do so. However, this behavior is unlike that of human artists that change their style as times go by and seldom return to the initial point. We investigate a situation where VAEs are requested to sample from a probability measure described by some empirical set. Based on recent works on Radon-Sobolev statistical distances, we pro-pose a numerical paradigm, to be used in conjunction with a generative algorithm, that satisﬁes the two following requirements: the objects created do not repeat and evolve to ﬁll the entire target probability measure
Ill-posedness of quintic fourth order Schr\"odinger equation

We prove that the solution map, associated to the quintic fourth order nonlinear Schr\"odinger equation, exhibits the norm inflation phenomenon at every point in the Sobolev spaces of super-critical regularity. Indeed, we prove this result separately in the cases of negative and of positive regularity. In the negative regularity case, we prove the result for both the defocusing and focusing equations: in the one dimensional case, the associated solution map exhibits norm inflation in Sobolev spaces of super-critical regularity (including the critical index); in the higher dimensional case, the solution map exhibits the same phenomenon in spaces of negative regularity. Meanwhile, in the case of positive regularity, we prove the result for the defocusing equation in dimensions $d=3,4,5$. Our proofs are based on the"high-to-low"and"low-to-high"frequency cascades respectively.
Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data

Selecting suitable architecture parameters and training hyperparameters is essential for enhancing machine learning (ML) model performance. Several recent empirical studies conduct large-scale correlational analysis on neural networks (NNs) to search for effective \emph{generalization metrics} that can guide this type of model selection. Effective metrics are typically expected to correlate strongly with test performance. In this paper, we expand on prior analyses by examining generalization-metric-based model selection with the following objectives: (i) focusing on natural language processing (NLP) tasks, as prior work primarily concentrates on computer vision (CV) tasks; (ii) considering metrics that directly predict \emph{test error} instead of the \emph{generalization gap}; (iii) exploring metrics that do not need access to data to compute. From these objectives, we are able to provide the first model selection results on large pretrained Transformers from Huggingface using generalization metrics. Our analyses consider (I) hundreds of Transformers trained in different settings, in which we systematically vary the amount of data, the model size and the optimization hyperparameters, (II) a total of 51 pretrained Transformers from eight families of Huggingface NLP models, including GPT2, BERT, etc., and (III) a total of 28 existing and novel generalization metrics. Despite their niche status, we find that metrics derived from the heavy-tail (HT) perspective are particularly useful in NLP tasks, exhibiting stronger correlations than other, more popular metrics. To further examine these metrics, we extend prior formulations relying on power law (PL) spectral distributions to exponential (EXP) and exponentially-truncated power law (E-TPL) families.
Energy awareness in low precision neural networks

Power consumption is a major obstacle in the deployment of deep neural networks (DNNs) on end devices. Existing approaches for reducing power consumption rely on quite general principles, including avoidance of multiplication operations and aggressive quantization of weights and activations. However, these methods do not take into account the precise power consumed by each module in the network, and are therefore not optimal. In this paper we develop accurate power consumption models for all arithmetic operations in the DNN, under various working conditions. We reveal several important factors that have been overlooked to date. Based on our analysis, we present PANN (power-aware neural network), a simple approach for approximating any full-precision network by a low-power fixed-precision variant. Our method can be applied to a pre-trained network, and can also be used during training to achieve improved performance. In contrast to previous methods, PANN incurs only a minor degradation in accuracy w.r. t. the full-precision version of the network, even when working at the power-budget of a 2-bit quantized variant. In addition, our scheme enables to seamlessly traverse the power-accuracy trade-off at deployment time, which is a major advantage over existing quantization methods that are constrained to specific bit widths.
Message Passing Neural PDE Solvers

The numerical solution of partial differential equations (PDEs) is difficult, having led to a century of research so far. Recently, there have been pushes to build neural--numerical hybrid solvers, which piggy-backs the modern trend towards fully end-to-end learned systems. Most works so far can only generalize over a subset of properties to which a generic solver would be faced, including: resolution, topology, geometry, boundary conditions, domain discretization regularity, dimensionality, etc. In this work, we build a solver, satisfying these properties, where all the components are based on neural message passing, replacing all heuristically designed components in the computation graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. In order to encourage stability in training autoregressive models, we put forward a method that is based on the principle of zero-stability, posing stability as a domain adaptation problem. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.