Miracle-dz/UniFAR-FTUs huggingface dataset:
Deep Internal Learning: Deep Learning from a Single Input

Deep learning in general focuses on training a neural network from large labeled datasets. Yet, in many cases there is value in training a network just from the input at hand. This may involve training a network from scratch using a single input or adapting an already trained network to a provided input example at inference time. This survey paper aims at covering deep internal-learning techniques that have been proposed in the past few years for these two important directions. While our main focus will be on image processing problems, most of the approaches that we survey are derived for general signals (vectors with recurring patterns that can be distinguished from noise) and are therefore applicable to other modalities. We believe that the topic of internal-learning is very important in many signal and image processing problems where training data is scarce and diversity is large on the one hand, and on the other, there is a lot of structure in the data that can be exploited.
CLASS-M: Adaptive stain separation-based contrastive learning with pseudo-labeling for histopathological image classification

Histopathological image classification is an important task in medical image analysis. Recent approaches generally rely on weakly supervised learning due to the ease of acquiring case-level labels from pathology reports. However, patch-level classification is preferable in applications where only a limited number of cases are available or when local prediction accuracy is critical. On the other hand, acquiring extensive datasets with localized labels for training is not feasible. In this paper, we propose a semi-supervised patch-level histopathological image classification model, named CLASS-M, that does not require extensively labeled datasets. CLASS-M is formed by two main parts: a contrastive learning module that uses separated Hematoxylin and Eosin images generated through an adaptive stain separation process, and a module with pseudo-labels using MixUp. We compare our model with other state-of-the-art models on two clear cell renal cell carcinoma datasets. We demonstrate that our CLASS-M model has the best performance on both datasets. Our code is available at github.com/BzhangURU/Paper_CLASS-M/tree/main
"It doesn't tell me anything about how my data is used": User Perceptions of Data Collection Purposes

Data collection purposes and their descriptions are presented on almost all privacy notices under the GDPR, yet there is a lack of research focusing on how effective they are at informing users about data practices. We fill this gap by investigating users' perceptions of data collection purposes and their descriptions, a crucial aspect of informed consent. We conducted 23 semi-structured interviews with European users to investigate user perceptions of six common purposes (Strictly Necessary, Statistics and Analytics, Performance and Functionality, Marketing and Advertising, Personalized Advertising, and Personalized Content) and identified elements of an effective purpose name and description. We found that most purpose descriptions do not contain the information users wish to know, and that participants preferred some purpose names over others due to their perceived transparency or ease of understanding. Based on these findings, we suggest how the framing of purposes can be improved toward meaningful informed consent.
A space-time continuous and coercive formulation for the wave equation

We propose a new space-time variational formulation for wave equation initial-boundary value problems. The key property is that the formulation is coercive (sign-definite) and continuous in a norm stronger than $H^1(Q)$, $Q$ being the space-time cylinder. Coercivity holds for constant-coefficient impedance cavity problems posed in star-shaped domains, and for a class of impedance-Dirichlet problems. The formulation is defined using simple Morawetz multipliers and its coercivity is proved with elementary analytical tools, following earlier work on the Helmholtz equation. The formulation can be stably discretised with any $H^2(Q)$-conforming discrete space, leading to quasi-optimal space-time Galerkin schemes. Several numerical experiments show the excellent properties of the method.
Electronic band structure of Sb2Te3

Here we report on Landau level spectroscopy of an epitaxially grown thin film of the topological insulator Sb2Te3, complemented by ellipsometry and magneto-transport measurements. The observed response suggests that Sb2Te3 is a direct-gap semiconductor with the fundamental band gap located at the \Gamma point, or along the trigonal axis, and its width reaches Eg = 190 meV at low temperatures. Our data also indicate the presence of other low-energy extrema with a higher multiplicity in both the conduction and valence bands. The conclusions based on our experimental data are confronted with and to a great extent corroborated by the electronic band structure calculated using the GW method.
Perseus: Removing Energy Bloat from Large Model Training

Training large AI models on numerous GPUs consumes a massive amount of energy. We observe that not all energy consumed during training directly contributes to end-to-end training throughput, and a significant portion can be removed without slowing down training, which we call energy bloat. In this work, we identify two independent sources of energy bloat in large model training, intrinsic and extrinsic, and propose Perseus, a unified optimization framework that mitigates both. Perseus obtains the"iteration time-energy"Pareto frontier of any large model training job using an efficient iterative graph cut-based algorithm and schedules energy consumption of its forward and backward computations across time to remove intrinsic and extrinsic energy bloat. Evaluation on large models like GPT-3 and Bloom shows that Perseus reduces energy consumption of large model training by up to 30%, enabling savings otherwise unobtainable before.
Task-Agnostic Privacy-Preserving Representation Learning for Federated Learning Against Attribute Inference Attacks

Federated learning (FL) has been widely studied recently due to its property to collaboratively train data from different devices without sharing the raw data. Nevertheless, recent studies show that an adversary can still be possible to infer private information about devices' data, e.g., sensitive attributes such as income, race, and sexual orientation. To mitigate the attribute inference attacks, various existing privacy-preserving FL methods can be adopted/adapted. However, all these existing methods have key limitations: they need to know the FL task in advance, or have intolerable computational overheads or utility losses, or do not have provable privacy guarantees. We address these issues and design a task-agnostic privacy-preserving presentation learning method for FL (TAPPFL) against attribute inference attacks. TAPPFL is formulated via information theory. Specifically, TAPPFL has two mutual information goals, where one goal learns task-agnostic data representations that contain the least information about the private attribute in each device's data, and the other goal ensures the learnt data representations include as much information as possible about the device data to maintain FL utility. We also derive privacy guarantees of TAPPFL against worst-case attribute inference attacks, as well as the inherent tradeoff between utility preservation and privacy protection. Extensive results on multiple datasets and applications validate the effectiveness of TAPPFL to protect data privacy, maintain the FL utility, and be efficient as well. Experimental results also show that TAPPFL outperforms the existing defenses.
Mean-field limits for Consensus-Based Optimization and Sampling

For algorithms based on interacting particle systems that admit a mean-field description, convergence analysis is often more accessible at the mean-field level. In order to transpose convergence results obtained at the mean-field level to the finite ensemble size setting, it is desirable to show that the particle dynamics converge in an appropriate sense to the corresponding mean-field dynamics. In this paper, we prove quantitative mean-field limit results for two related interacting particle systems: Consensus-Based Optimization and Consensus-Based Sampling. Our approach extends Sznitman's classical argument: in order to circumvent issues related to the lack of global Lipschitz continuity of the coefficients, we discard an event of small probability, the contribution of which is controlled using moment estimates for the particle systems.
Investigation into the Training Dynamics of Learned Optimizers

Optimization is an integral part of modern deep learning. Recently, the concept of learned optimizers has emerged as a way to accelerate this optimization process by replacing traditional, hand-crafted algorithms with meta-learned functions. Despite the initial promising results of these methods, issues with stability and generalization still remain, limiting their practical use. Moreover, their inner workings and behavior under different conditions are not yet fully understood, making it difficult to come up with improvements. For this reason, our work examines their optimization trajectories from the perspective of network architecture symmetries and parameter update distributions. Furthermore, by contrasting the learned optimizers with their manually designed counterparts, we identify several key insights that demonstrate how each approach can benefit from the strengths of the other.
Anatomically Constrained Implicit Face Models

Coordinate based implicit neural representations have gained rapid popularity in recent years as they have been successfully used in image, geometry and scene modeling tasks. In this work, we present a novel use case for such implicit representations in the context of learning anatomically constrained face models. Actor specific anatomically constrained face models are the state of the art in both facial performance capture and performance retargeting. Despite their practical success, these anatomical models are slow to evaluate and often require extensive data capture to be built. We propose the anatomical implicit face model; an ensemble of implicit neural networks that jointly learn to model the facial anatomy and the skin surface with high-fidelity, and can readily be used as a drop in replacement to conventional blendshape models. Given an arbitrary set of skin surface meshes of an actor and only a neutral shape with estimated skull and jaw bones, our method can recover a dense anatomical substructure which constrains every point on the facial surface. We demonstrate the usefulness of our approach in several tasks ranging from shape fitting, shape editing, and performance retargeting.
Constraining ultralight bosons in dwarf spheroidal galaxies with a radially varying anisotropy

Axions, and axion-like particles, have come back into fashion in the last decades as a possible solution to the galactic-scale crisis suffered by the cold dark matter model. In the framework of the wave Dark Matter model, we have carried out a Jeans Analysis on eight dwarf spheroidal galaxies that are orbiting around the Milky Way, and we have constrained the boson mass. Differently to a previous analysis, we adopted an anisotropy parameter that varies with the distance from the centre of the galaxy to assess whether this assumption would help to resolve, or at least alleviate, the well-known tension with the value of the boson mass favoured by the cosmological analysis. Our results indicate that, differently to what happens in ultra-faint dwarf galaxies, such a tension cannot be lifted introducing a variable anisotropy parameter, leaving as a possible solution the existence of additional axion or axion-like particles with higher masses as naturally predicted in the Axiverse.
Investigating lipid bilayer self-assembly and formation of ripple phase: Insights from a coarse-grained implicit solvent model

In this study, we present a comprehensive exploration of the formation of different phases in lipid molecules using a coarse-grained implicit solvent model, where each lipid molecule is presented as a rigid, three-bead, rod-like structure. Our study not only successfully replicates the spontaneous self-assembly of lipid bilayers but also elucidates the intricate phase transitions between the gel phase, the liquid phase, and the elusive ripple phase (Pβ). Specifically, we uncover the pivotal role of molecular rigidity in promoting the formation of the ripple phase. The significance of our findings lies in their potential to reshape our understanding of lipid bilayer dynamics and phase transitions. By shedding light on the ripple phase, a phase previously difficult to simulate convincingly, the insights gained from this study have the potential to guide future investigations into the behavior of biological membranes and their role in cellular processes.
Operations on the de Rham cohomology of Poisson and Jacobi manifolds

We prove that the (homotopy) hypercommutative algebra structure on the de Rham cohomology of a Poisson or Jacobi manifold defined by several authors is (homotopically) trivial, i.e. it reduces to the underlying (homotopy) commutative algebra structure. We do so by showing that the DG operads which codify the algebraic structure on the de Rham complex of Poisson and Jacobi manifolds, generated by the exterior product and the interior products with the structure polyvector fields, are quasi-isomorphic to the commutative suboperad. Hence, there is no hope to endow the de Rham cohomology of such manifolds with any (higher) structure beyond the well-known (homotopy) commutative algebra structure, which exists for any smooth manifold. We proceed similarly with the commutative Batalin-Vilkovisky-infinity algebra structure on the de Rham complex of a generalized Poisson supermanifold.
Revisiting the convergence of the perturbative QCD expansions based on conformal mapping of the Borel plane

The difference between fixed-order (FO) and contour-improved (CI) formulations of QCD perturbation theory limits the precision of the strong coupling determined from the hadronic decay of the $\tau$ lepton. Recently, several attempts to understand the mathematical origin of the difference and to solve it by subtracting the dominant infrared renormalon divergence have been made. Motivated by these studies, we review in this paper an improved perturbative QCD expansion, defined some time ago, which also exploits the renormalons by means of a suitable conformal mapping of the Borel plane. In particular, we revisit the convergence of the new expansion, by completing the proof presented in a previous paper and showing that the domain of convergence is larger than stated before. We also check the validity of the convergence conditions for the Adler function and the CI and FO expansions of the $\tau$ hadronic spectral function moments, and compare the approach based on conformal mapping with recent solutions to the CIPT-FOPT discrepancy proposed in the literature.
Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models

Despite recent significant strides achieved by diffusion-based Text-to-Image (T2I) models, current systems are still less capable of ensuring decent compositional generation aligned with text prompts, particularly for the multi-object generation. This work illuminates the fundamental reasons for such misalignment, pinpointing issues related to low attention activation scores and mask overlaps. While previous research efforts have individually tackled these issues, we assert that a holistic approach is paramount. Thus, we propose two novel objectives, the Separate loss and the Enhance loss, that reduce object mask overlaps and maximize attention scores, respectively. Our method diverges from conventional test-time-adaptation techniques, focusing on finetuning critical parameters, which enhances scalability and generalizability. Comprehensive evaluations demonstrate the superior performance of our model in terms of image realism, text-image alignment, and adaptability, notably outperforming prominent baselines. Ultimately, this research paves the way for T2I diffusion models with enhanced compositional capacities and broader applicability.
AI capabilities can be significantly improved without expensive retraining

State-of-the-art AI systems can be significantly improved without expensive retraining via"post-training enhancements"-techniques applied after initial training like fine-tuning the system to use a web browser. We review recent post-training enhancements, categorizing them into five types: tool-use, prompting methods, scaffolding, solution selection, and data generation. Different enhancements improve performance on different tasks, making it hard to compare their significance. So we translate improvements from different enhancements into a common currency, the compute-equivalent gain: how much additional training compute would be needed to improve performance by the same amount as the enhancement. Our non-experimental work shows that post-training enhancements have significant benefits: most surveyed enhancements improve benchmark performance by more than a 5x increase in training compute, some by more than 20x. Post-training enhancements are relatively cheap to develop: fine-tuning costs are typically<1% of the original training cost. Governing the development of capable post-training enhancements may be challenging because frontier models could be enhanced by a wide range of actors.
Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection

This work studies the recently proposed challenging and practical Multi-class Unsupervised Anomaly Detection (MUAD) task, which only requires normal images for training while simultaneously testing both normal/anomaly images for multiple classes. Existing reconstruction-based methods typically adopt pyramid networks as encoders/decoders to obtain multi-resolution features, accompanied by elaborate sub-modules with heavier handcraft engineering designs for more precise localization. In contrast, a plain Vision Transformer (ViT) with simple architecture has been shown effective in multiple domains, which is simpler, more effective, and elegant. Following this spirit, this paper explores plain ViT architecture for MUAD. Specifically, we abstract a Meta-AD concept by inducing current reconstruction-based methods. Then, we instantiate a novel and elegant plain ViT-based symmetric ViTAD structure, effectively designed step by step from three macro and four micro perspectives. In addition, this paper reveals several interesting findings for further exploration. Finally, we propose a comprehensive and fair evaluation benchmark on eight metrics for the MUAD task. Based on a naive training recipe, ViTAD achieves state-of-the-art (SoTA) results and efficiency on the MVTec AD and VisA datasets without bells and whistles, obtaining 85.4 mAD that surpasses SoTA UniAD by +3.0, and only requiring 1.1 hours and 2.3G GPU memory to complete model training by a single V100 GPU. Source code, models, and more results are available at https://zhangzjn.github. io/projects/ViTAD.
Class-Prototype Conditional Diffusion Model for Continual Learning with Generative Replay

Mitigating catastrophic forgetting is a key hurdle in continual learning. Deep Generative Replay (GR) provides techniques focused on generating samples from prior tasks to enhance the model’s memory capabilities. With the progression in generative AI, generative models have advanced from Generative Adversarial Networks (GANs) to the more recent Diffusion Models (DMs). A major issue is the deterioration in the quality of generated data compared to the original, as the generator continuously self-learns from its outputs. This degradation can lead to the potential risk of catastrophic forgetting occurring in the classifier. To address this, we propose the Class-Prototype Conditional Diffusion Model (CPDM), a GR-based approach for continual learning that enhances image quality in generators and thus reduces catastrophic forgetting in classifiers. The cornerstone of CPDM is a learnable class-prototype that captures the core characteristics of images in a given class. This prototype, integrated into the diffusion model’s denoising process, ensures the generation of high-quality images. It maintains its effectiveness for old tasks even when new tasks are introduced, preserving image generation quality and reducing the risk of catastrophic forgetting in classifiers. Our empirical studies on diverse datasets demonstrate that our proposed method significantly outperforms existing state-of-the-art models, highlighting its exceptional ability to preserve image quality and enhance the model’s memory retention. Code is available at https://github.com/ dnkhanh45/CPDM .
Gravitational atoms in the braneworld scenario

General relativity (GR) may be modified by adding an extra warped noncompact spatial dimension such that it is indistinguishable from GR as far as local tests of gravity are concerned. However, such a modified theory of gravity should have intriguing consequences on various aspects of black holes which can help us probe how the presence of an extra dimension affects gravity in the strong field regime, and enable us to distinguish it from GR. In this context, we have studied massive scalar perturbations around four-dimensional rotating black holes in the Randall Sundrum II braneworld scenario. These black holes are endowed with a tidal charge that contains information pertaining to the extra spatial dimension in the braneworld model. Such black hole spacetimes are also noteworthy because they permit the black hole's rotation parameter to exceed unity, a possibility strictly forbidden in GR. Consequently, they offer valuable insights into exploring the repercussions of modifications to Einstein's theory through possible future observations. Our approach involves the numerical solution of the perturbed field equations using the method of continued fractions. First we ascertain the quasinormal mode spectra of the rotating braneworld black hole. We then thoroughly investigate the existence of quasibound states and the associated superradiant instability. Such a superradiant cloud of bosons around the black hole is called a gravitational atom and they are invaluable observational probes of ultralight bosonic particles predicted by various extensions of the Standard Model of particle physics. In comparison to the four-dimensional Kerr black hole, we report distinctive signatures of the tidal charge and the rotation parameter, which manifest as signals of the extra dimension, on both the fundamental quasinormal mode and the formation of gravitational atoms.
Leveraging Generative Language Models for Weakly Supervised Sentence Component Analysis in Video-Language Joint Learning

A thorough comprehension of textual data is a fundamental element in multi-modal video analysis tasks. However, recent works have shown that the current models do not achieve a comprehensive understanding of the textual data during the training for the target downstream tasks. Orthogonal to the previous approaches to this limitation, we postulate that understanding the significance of the sentence components according to the target task can potentially enhance the performance of the models. Hence, we utilize the knowledge of a pre-trained large language model (LLM) to generate text samples from the original ones, targeting specific sentence components. We propose a weakly supervised importance estimation module to compute the relative importance of the components and utilize them to improve different video-language tasks. Through rigorous quantitative analysis, our proposed method exhibits significant improvement across several video-language tasks. In particular, our approach notably enhances video-text retrieval by a relative improvement of 8.3\% in video-to-text and 1.4\% in text-to-video retrieval over the baselines, in terms of R@1. Additionally, in video moment retrieval, average mAP shows a relative improvement ranging from 2.0\% to 13.7 \% across different baselines.