Miracle-dz/UniFAR-FTUs huggingface dataset:
Adversarial Purification of Information Masking

Adversarial attacks meticulously generate minuscule, imperceptible perturbations to images to deceive neural networks. Counteracting these, adversarial purification methods seek to transform adversarial input samples into clean output images to defend against adversarial attacks. Nonetheless, extent generative models fail to effectively eliminate adversarial perturbations, yielding less-than-ideal purification results. We emphasize the potential threat of residual adversarial perturbations to target models, quantitatively establishing a relationship between perturbation scale and attack capability. Notably, the residual perturbations on the purified image primarily stem from the same-position patch and similar patches of the adversarial sample. We propose a novel adversarial purification approach named Information Mask Purification (IMPure), aims to extensively eliminate adversarial perturbations. To obtain an adversarial sample, we first mask part of the patches information, then reconstruct the patches to resist adversarial perturbations from the patches. We reconstruct all patches in parallel to obtain a cohesive image. Then, in order to protect the purified samples against potential similar regional perturbations, we simulate this risk by randomly mixing the purified samples with the input samples before inputting them into the feature extraction network. Finally, we establish a combined constraint of pixel loss and perceptual loss to augment the model's reconstruction adaptability. Extensive experiments on the ImageNet dataset with three classifier models demonstrate that our approach achieves state-of-the-art results against nine adversarial attack methods. Implementation code and pre-trained weights can be accessed at \textcolor{blue}{https://github.com/NoWindButRain/IMPure}.
Introducing SSBD+ Dataset with a Convolutional Pipeline for Detecting Self-Stimulatory Behaviours in Children Using Raw Videos

Conventionally, evaluation for the diagnosis of Autism spectrum disorder is done by a trained specialist through questionnaire-based formal assessments and by observation of behavioral cues under various settings to capture the early warning signs of autism. These evaluation techniques are highly subjective and their accuracy relies on the experience of the specialist. In this regard, machine learning-based methods for automated capturing of early signs of autism from the recorded videos of the children is a promising alternative. In this paper, the authors propose a novel pipelined deep learning architecture to detect certain self-stimulatory behaviors that help in the diagnosis of autism spectrum disorder (ASD). The authors also supplement their tool with an augmented version of the Self Stimulatory Behavior Dataset (SSBD) and also propose a new label in SSBD Action detection: no-class. The deep learning model with the new dataset is made freely available for easy adoption to the researchers and developers community. An overall accuracy of around 81% was achieved from the proposed pipeline model that is targeted for real-time and hands-free automated diagnosis. All of the source code, data, licenses of use, and other relevant material is made freely available in [1]. Clinical relevance- Detection of Self-Stimulatory behaviors from recorded videos forms a key step towards the development of automated and cost-effective technology for screening early diagnosis and tracking of developmental disorders from The authors thank the psychiatrists at Bubbles Center for Autism, India for providing us with annotations for the videos in the SSBD+ dataset. The authors acknowledge the support and the research grant from IBM GUP.
Benchmarking Robustness of Text-Image Composed Retrieval

Text-image composed retrieval aims to retrieve the target image through the composed query, which is specified in the form of an image plus some text that describes desired modifications to the input image. It has recently attracted attention due to its ability to leverage both information-rich images and concise language to precisely express the requirements for target images. However, the robustness of these approaches against real-world corruptions or further text understanding has never been studied. In this paper, we perform the first robustness study and establish three new diversified benchmarks for systematic analysis of text-image composed retrieval against natural corruptions in both vision and text and further probe textural understanding. For natural corruption analysis, we introduce two new large-scale benchmark datasets, CIRR-C and FashionIQ-C for testing in open domain and fashion domain respectively, both of which apply 15 visual corruptions and 7 textural corruptions. For textural understanding analysis, we introduce a new diagnostic dataset CIRR-D by expanding the original raw data with synthetic data, which contains modified text to better probe textual understanding ability including numerical variation, attribute variation, object removal, background variation, and fine-grained evaluation. The code and benchmark datasets are available at https://github.com/SunTongtongtong/Benchmark-Robustness-Text-Image-Compose-Retrieval.
Optimality conditions for bilevel programs via Moreau envelope reformulation

For bilevel programs with a convex lower level program, the classical approach replaces the lower level program with its Karush-Kuhn-Tucker condition and solve the resulting mathematical program with complementarity constraint (MPCC). It is known that when the set of lower level multipliers is not unique, MPCC may not be equivalent to the original bilevel problem, and many MPCC-tailored constraint qualifications do not hold. In this paper, we study bilevel programs where the lower level is generalized convex. Applying the equivalent reformulation via Moreau envelope, we derive new directional optimality conditions. Even in the nondirectional case, the new optimality condition is stronger than the strong stationarity for the corresponding MPCC.
Spectral Moment Formulae for $GL(3)\times GL(2)$ $L$-functions III: The Twisted Case

This is a sequel to our previous articles \cite{Kw23, Kw23a+}. In this work, we apply recent techniques that fall under the banner of `Period Reciprocity' to study moments of $GL(3)\times GL(2)$ $L$-functions in the non-archimedean aspects, with a view towards the `Twisted Moment Conjectures' formulated by CFKRS.
The Vertical Component of Epineutral Diffusion and the Dianeutral Component of Horizontal Diffusion

Abstract Observations of the time-mean thermohaline state of the world oceans are used to identify temperature gradients (and implied diffusive heat fluxes) both horizontally and on density surfaces. Two types of density surface are used: neutral and isopycnal, with the latter giving significantly different (and poorer) results. Along-neutral-surface (epineutral) heat fluxes are estimated to have a vertical component that is upward and strong in the upper 1500 m of the extratropical oceans, and weakly downward elsewhere. Horizontal heat diffusion has a dianeutral component with a similar pattern of strong and weak fluxes, although almost all fluxes are toward water of greater density.
TulipaProfileFitting.jl: A Julia package for fitting renewable energy time series profiles

This paper introduces the TulipaProfileFitting.jl package, a tool developed in Julia for generating renewable energy profiles that fit a specified capacity factor. It addresses the limitations of naive methods in adjusting existing profiles to match improved technology efficiency, particularly in scenarios lacking detailed weather data or technology specifications. By formulating the problem mathematically, the package provides a computationally efficient solution for creating realistic renewable energy profiles based on existing data. It ensures that the adjusted profiles realistically reflect the improvements in technology efficiency, making it an essential tool for energy modellers in analyzing $CO_2$-neutral energy systems.
Topological Thermal Hall Conductance of Even Denominator Fractional States

The even denominator fractional quantum Hall (FQH) states $\nu=5/2$ and $\nu=7/2$, have been long predicted to host non-abelian quasiparticles (QPs). The presence of energy-carrying neutral modes cripples customary conductance measurements and thus motivates thermal transport measurements, which already proved to be sensitive to all energy-carrying modes. Each state has a different capacity to carry quanta of heat - as expressed by the so-called: 'central charge' - identifying the state's topological order. While the 'two-terminal' thermal conductance measurements identified the topological orders of abelian and non-abelian QH states, they are prone to partial thermal equilibration among counter-propagating modes. Here, we report a 'four-terminal' thermal Hall conductance measurement, which separately measures the heat carried by the downstream and upstream chiral modes. This measurement is insensitive to thermal equilibration among modes. We verify that the $\nu=5/2$ and $\nu=7/2$ states are non-abelian, supporting a single upstream Majorana mode, thus obeying the Particle-Hole Pfaffian topological order. While current numerical works predict a different central charge, this contribution should motivate further theoretical work.
Wavelength-multiplexed Multi-mode EUV Reflection Ptychography based on Automatic-Differentiation

Ptychographic extreme ultraviolet (EUV) diffractive imaging has emerged as a promising candidate for the next-generation metrology solutions in the semiconductor industry, as it can image wafer samples in reflection geometry at the nanoscale. This technique has surged attention recently, owing to the significant progress in high-harmonic generation (HHG) EUV sources and advancements in both hardware and software for computation. In this study, a novel algorithm is introduced and tested, which enables wavelength-multiplexed reconstruction that enhances the measurement throughput and introduces data diversity, allowing the accurate characterisation of sample structures. To tackle the inherent instabilities of the HHG source, a modal approach was adopted, which represents the cross-density function of the illumination by a series of mutually incoherent and independent spatial modes. The proposed algorithm was implemented on a mainstream machine learning platform, which leverages automatic differentiation to manage the drastic growth in model complexity and expedites the computation using GPU acceleration. By optimising over 200 million parameters, we demonstrate the algorithm's capacity to accommodate experimental uncertainties and achieve a resolution approaching the diffraction limit in reflection geometry. The reconstruction of wafer samples with 20-nm heigh patterned gold structures on a silicon substrate highlights our ability to handle complex physical interrelations involving a multitude of parameters. These results establish ptychography as an efficient and accurate metrology tool.
The Rise of the AI Co-Pilot: Lessons for Design from Aviation and Beyond

The fast pace of advances in AI promises to revolutionize various aspects of knowledge work, extending its influence to daily life and professional fields alike. We advocate for a paradigm where AI is seen as a collaborative co-pilot, working under human guidance rather than as a mere tool. Drawing from relevant research and literature in the disciplines of Human-Computer Interaction and Human Factors Engineering, we highlight the criticality of maintaining human oversight in AI interactions. Reflecting on lessons from aviation, we address the dangers of over-relying on automation, such as diminished human vigilance and skill erosion. Our paper proposes a design approach that emphasizes active human engagement, control, and skill enhancement in the AI partnership, aiming to foster a harmonious, effective, and empowering human-AI relationship. We particularly call out the critical need to design AI interaction capabilities and software applications to enable and celebrate the primacy of human agency. This calls for designs for human-AI partnership that cede ultimate control and responsibility to the human user as pilot, with the AI co-pilot acting in a well-defined supporting role.
Quantifying equilibrium pressure-gradient turbulent boundary layers via a symmetry approach

We propose a theory for predicting the mean velocity and Reynolds shear and normal stresses profiles in the wake region of equilibrium adverse pressure-gradient (PG, APG) turbulent boundary layers (TBLs). Firstly, we explore the PG-induced dilation-symmetry-breaking of the total stress $\tau^+$ to construct a modified defect power law for $\tau^+$. Crucially, a PG stress $P_0^+$ is identified, which quantifies the APG-induced total-stress overshoot and is proportional to the Clauser PG parameter $\beta$. The wall-normal location with peak stress is predicted. The total stress profiles with arbitrary $\beta$ are transformed into an invariant profile, which is the ultimate state of the total stress at infinite $\beta$. This transformation is equivalent to the outer scaling of the Reynolds shear stress recently-proposed by Wei&Knopp (JFM, 2023). The Reynolds normal stresses are predicted accordingly based on the similarity of the Reynolds shear and normal stresses in the wake region. Secondly, a defect power law is proposed for the stress and kinetic energy lengths in the wake region. Two critical parameters in the defect power law are identified to depend on $\beta$ and determine the length profiles. With the total stress and stress length models, the streamwise mean-velocity profile is predicted. Especially, an invariant mean velocity profile is derived, which describes the ultimate state of the mean velocity in the wake region at infinite $\beta$. This invariant profile is also equivalent to the outer scaling of Wei&Knopp. The theory also predicts the variation of the Coles' wake parameter $\Pi$ with $\beta$, in close agreement with the empirical relation that correlates hundreds of experimental data. The predictions are validated with five published DNS, LES, and experimental databases on the equilibrium APG TBLs.
Compositional Zero-shot Learning via Progressive Language-based Observations

Compositional zero-shot learning aims to recognize unseen state-object compositions by leveraging known primitives (state and object) during training. However, effectively modeling interactions between primitives and generalizing knowledge to novel compositions remains a perennial challenge. There are two key factors: object-conditioned and state-conditioned variance, i.e., the appearance of states (or objects) can vary significantly when combined with different objects (or states). For instance, the state"old"can signify a vintage design for a"car"or an advanced age for a"cat". In this paper, we argue that these variances can be mitigated by predicting composition categories based on pre-observed primitive. To this end, we propose Progressive Language-based Observations (PLO), which can dynamically determine a better observation order of primitives. These observations comprise a series of concepts or languages that allow the model to understand image content in a step-by-step manner. Specifically, PLO adopts pre-trained vision-language models (VLMs) to empower the model with observation capabilities. We further devise two variants: 1) PLO-VLM: a two-step method, where a pre-observing classifier dynamically determines the observation order of two primitives. 2) PLO-LLM: a multi-step scheme, which utilizes large language models (LLMs) to craft composition-specific prompts for step-by-step observing. Extensive ablations on three challenging datasets demonstrate the superiority of PLO compared with state-of-the-art methods, affirming its abilities in compositional recognition.
Pseudo-likelihood Estimators for Graphical Models: Existence and Uniqueness

Graphical and sparse (inverse) covariance models have found widespread use in modern sample-starved high dimensional applications. A part of their wide appeal stems from the significantly low sample sizes required for the existence of estimators, especially in comparison with the classical full covariance model. For undirected Gaussian graphical models, the minimum sample size required for the existence of maximum likelihood estimators had been an open question for almost half a century, and has been recently settled. The very same question for pseudo-likelihood estimators has remained unsolved ever since their introduction in the '70s. Pseudo-likelihood estimators have recently received renewed attention as they impose fewer restrictive assumptions and have better computational tractability, improved statistical performance, and appropriateness in modern high dimensional applications, thus renewing interest in this longstanding problem. In this paper, we undertake a comprehensive study of this open problem within the context of the two classes of pseudo-likelihood methods proposed in the literature. We provide a precise answer to this question for both pseudo-likelihood approaches and relate the corresponding solutions to their Gaussian counterpart.
Operator-Valued Hardy spaces and BMO Spaces on Spaces of Homogeneous Type

Let $\mathcal{M}$ be a von Neumann algebra equipped with a normal semifinite faithful trace, $(\mathbb{X},\,d,\,\mu)$ be a space of homogeneous type in the sense of Coifman and Weiss, and $\mathcal{N}=L_\infty(\mathbb{X})\overline{\otimes}\mathcal{M}$. In this paper, we introduce and then conduct a systematic study on the operator-valued Hardy space $\mathcal{H}_p(\mathbb{X},\,\mathcal{M})$ for all $1\leq p<\infty$ and operator-valued BMO space $\mathcal{BMO}(\mathbb{X},\,\mathcal{M})$. The main results of this paper include $H_1$--$BMO$ duality theorem, atomic decomposition of $\mathcal{H}_1(\mathbb{X},\,\mathcal{M})$, interpolation between these Hardy spaces and BMO spaces, and equivalence between mixture Hardy spaces and $L_p$-spaces. %Compared with the communcative results, the novelty of this article is that $\mu$ is not assumed to satisfy the reverse double condition. %The approaches we develop bypass the use of harmonicity of infinitesimal generator, which allows us to extend Mei's seminal work \cite{m07} to a broader setting. %Our results extend Mei's seminal work \cite{m07} to a broader setting. In particular, without the use of non-commutative martingale theory as in Mei's seminal work \cite{m07}, we provide a direct proof for the interpolation theory. Moreover, under our assumption on Calder\'{o}n representation formula, these results are even new when going back to the commutative setting for spaces of homogeneous type which fails to satisfy reverse doubling condition. As an application, we obtain the $L_p(\mathcal{N})$-boundedness of operator-valued Calder\'{o}n-Zygmund operators.
Should We Learn Most Likely Functions or Parameters?

Standard regularized training procedures correspond to maximizing a posterior distribution over parameters, known as maximum a posteriori (MAP) estimation. However, model parameters are of interest only insomuch as they combine with the functional form of a model to provide a function that can make good predictions. Moreover, the most likely parameters under the parameter posterior do not generally correspond to the most likely function induced by the parameter posterior. In fact, we can re-parametrize a model such that any setting of parameters can maximize the parameter posterior. As an alternative, we investigate the benefits and drawbacks of directly estimating the most likely function implied by the model and the data. We show that this procedure leads to pathological solutions when using neural networks and prove conditions under which the procedure is well-behaved, as well as a scalable approximation. Under these conditions, we find that function-space MAP estimation can lead to flatter minima, better generalization, and improved robustness to overfitting.
InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser

Stylized text-to-image generation focuses on creating images from textual descriptions while adhering to a style specified by a few reference images. However, subtle style variations within different reference images can hinder the model from accurately learning the target style. In this paper, we propose InstaStyle, a novel approach that excels in generating high-fidelity stylized images with only a single reference image. Our approach is based on the finding that the inversion noise from a stylized reference image inherently carries the style signal, as evidenced by their non-zero signal-to-noise ratio. We employ DDIM inversion to extract this noise from the reference image and leverage a diffusion model to generate new stylized images from the"style"noise. Additionally, the inherent ambiguity and bias of textual prompts impede the precise conveying of style. To address this, we introduce a learnable style token via prompt refinement, which enhances the accuracy of the style description for the reference image. Qualitative and quantitative experimental results demonstrate that InstaStyle achieves superior performance compared to current benchmarks. Furthermore, our approach also showcases its capability in the creative task of style combination with mixed inversion noise.
Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learning

Many problems in Reinforcement Learning (RL) seek an optimal policy with large discrete multidimensional yet unordered action spaces; these include problems in randomized allocation of resources such as placements of multiple security resources and emergency response units, etc. A challenge in this setting is that the underlying action space is categorical (discrete and unordered) and large, for which existing RL methods do not perform well. Moreover, these problems require validity of the realized action (allocation); this validity constraint is often difficult to express compactly in a closed mathematical form. The allocation nature of the problem also prefers stochastic optimal policies, if one exists. In this work, we address these challenges by (1) applying a (state) conditional normalizing flow to compactly represent the stochastic policy -- the compactness arises due to the network only producing one sampled action and the corresponding log probability of the action, which is then used by an actor-critic method; and (2) employing an invalid action rejection method (via a valid action oracle) to update the base policy. The action rejection is enabled by a modified policy gradient that we derive. Finally, we conduct extensive experiments to show the scalability of our approach compared to prior methods and the ability to enforce arbitrary state-conditional constraints on the support of the distribution of actions in any state.