Miracle-dz/UniFAR-FTUs huggingface dataset:
All-Optical, Reconfigurable and Power Independent Neural Activation Function by Means of Phase Modulation

In this work, we present numerical results concerning an integrated photonic non-linear activation function that relies on a power independent, non-linear phase to amplitude conversion in a passive optical resonator. The underlying mechanism is universal to all optical filters, whereas here, simulations were based on micro-ring resonators (MRRs). Investigation revealed that the photonic neural node can be tuned to support a wide variety of continuous activation functions that are relevant to the neural network architectures, such as the sigmoid and the softplus functions. The proposed photonic node is numerically evaluated in the context of time delayed reservoir computing (TDRC) scheme, targeting the one-step ahead prediction of the Santa Fe series. The proposed phase to amplitude TDRC is benchmarked versus the conventional amplitude based TDRC, showcasing a performance boost by one order of magnitude.
Tilings of the Hyperbolic Space and Lipschitz Functions

We use a special tiling for the hyperbolic $d$-space $\mathbb{H}^d$ for $d=2,3,4$ to construct an (almost) explicit isomorphism between the Lipschitz-free space $\mathcal{F}(\mathbb{H}^d)$ and $\mathcal{F}(P)\oplus\mathcal{F}(\mathcal{N})$ where $P$ is a polytope in $\mathbb{R}^d$ and $\mathcal{N}$ a net in $\mathbb{H}^d$ coming from the tiling. This implies that the spaces $\mathcal{F}(\mathbb{H}^d)$ and $\mathcal{F}(\mathbb{R}^{d})\oplus \mathcal{F}(\mathcal{M})$ are isomorphic for every net $\mathcal{M}$ in $\mathbb{H}^d$. In particular, we obtain that, for $d=2,3,4$, $\mathcal{F}(\mathbb{H}^d)$ has a Schauder basis. Moreover, using a similar method, we also give an explicit isomorphism between $\mathrm{Lip}(\mathbb{H}^{d})$ and $\mathrm{Lip}(\mathbb{R}^d)$.
The Bethe Ansatz for the superconformal index with unequal angular momenta

A few years ago it was shown that the superconformal index of the $\mathcal{N}=4$ supersymmetric $SU(N)$ Yang-Mills theory in the large $N$ limit matches with the entropy of $1/16$-supersymmetric black holes in type IIB string theory on $AdS_5\times S^5$. In some cases, an even more detailed match between the two sides is possible. When the two angular momentum chemical potentials in the index are equal, the superconformal index can be written as a discrete sum of Bethe ansatz solutions, and it was shown that specific terms in this sum are in a one-to-one correspondence to stable black hole solutions, and that the matching can be extended to non-perturbative contributions from wrapped D3-branes. A Bethe ansatz approach to computing the superconformal index exists also when the ratio of the angular momentum chemical potentials is any rational number, but in those cases it involves a sum over a very large number of terms (growing exponentially with $N$). Benini et al showed that a specific one of these terms matches with the black hole, but the role of the other terms is not clear. In this paper we analyze some of the additional contributions to the index in the Bethe ansatz approach, and we find that their matching to the gravity side is much more complicated than in the case of equal chemical potentials. In particular, we find some contributions that are larger than the one which was found to match the black holes, so that they must cancel with other large contributions. We give some evidence that cancellations of this type are possible, but we leave a full understanding of how they work to the future.
From zero-mode intermittency to hidden symmetry in random scalar advection

The statistical behavior of scalars passively advected by random flows exhibits intermittency in the form of anomalous multiscaling, in many ways similar to the patterns commonly observed in incompressible high-Reynolds fluids. This similarity suggests a generic dynamical mechanism underlying intermittency, though its specific nature remains unclear. Scalar turbulence is framed in a linear setting that points towards a zero-mode scenario connecting anomalous scaling to the presence of statistical conservation laws; the duality is fully substantiated within Kraichnan theory of random flows. However, extending the zero-mode scenario to nonlinear settings faces formidable technical challenges. Here, we revisit the scalar problem in the light of a hidden symmetry scenario introduced in recent deterministic turbulence studies addressing the Sabra shell model and the Navier-Stokes equations. Hidden symmetry uses a rescaling strategy based entirely on symmetry considerations, transforming the original dynamics into a rescaled (hidden) system; It ultimately identifies the scaling exponents as the eigenvalues of a Perron-Frobenius operator acting on invariant measures of the rescaled equations. Considering a minimal shell model of scalar advection of the Kraichnan type that was previously studied by Biferale&Wirth, the present work extends the hidden symmetry approach to a stochastic setting, in order to explicitly contrast it with the zero-mode scenario. Our study indicates that the zero-mode scenario represents only one facet of intermittency, here prescribing the scaling exponents of even-order correlators. Besides, we argue that hidden symmetry provides a more generic mechanism, fully prescribing intermittency in terms of scaling anomalies, but also in terms of its multiplicative random nature and fusion rules required to explicitly compute zero-modes from first principles.
Confronting primordial black holes with LIGO-Virgo-KAGRA and the Einstein Telescope

The detection of gravitational waves (GWs) from binary black hole (BBH) coalescences by the LIGO-Virgo-KAGRA (LVK) Collaboration has raised fundamental questions about the genesis of these events. In this chapter, we explore the possibility that PBHs, proposed candidates for dark matter, may serve as the progenitors of the BBHs observed by LVK. Employing a Bayesian analysis, we constrain the PBH model using the LVK third GW Transient Catalog (GWTC-3), revealing that stellar-mass PBHs cannot dominate cold dark matter. Considering a mixed population of astrophysical black holes (ABHs) and PBHs, we determine that approximately $1/4$ of the detectable events in the GWTC-3 can be attributed to PBH binaries. We also forecast detectable event rate distributions for PBH and ABH binaries by the third-generation ground-based GW detectors, such as the Einstein Telescope, offering a potential avenue to distinguish PBHs from ABHs based on their distinct redshift evolutions.
Weakly Supervised Anomaly Detection via Knowledge-Data Alignment

Anomaly detection (AD) plays a pivotal role in numerous web-based applications, including malware detection, anti-money laundering, device failure detection, and network fault analysis. Most methods, which rely on unsupervised learning, are hard to reach satisfactory detection accuracy due to the lack of labels. Weakly Supervised Anomaly Detection (WSAD) has been introduced with a limited number of labeled anomaly samples to enhance model performance. Nevertheless, it is still challenging for models, trained on an inadequate amount of labeled data, to generalize to unseen anomalies. In this paper, we introduce a novel framework Knowledge-Data Alignment (KDAlign) to integrate rule knowledge, typically summarized by human experts, to supplement the limited labeled data. Specifically, we transpose these rules into the knowledge space and subsequently recast the incorporation of knowledge as the alignment of knowledge and data. To facilitate this alignment, we employ the Optimal Transport (OT) technique. We then incorporate the OT distance as an additional loss term to the original objective function of WSAD methodologies. Comprehensive experimental results on five real-world datasets demonstrate that our proposed KDAlign framework markedly surpasses its state-of-the-art counterparts, achieving superior performance across various anomaly types.
A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective

Vertical Federated Learning (VFL) is a federated learning paradigm where multiple participants, who share the same set of samples but hold different features, jointly train machine learning models. Although VFL enables collaborative machine learning without sharing raw data, it is still susceptible to various privacy threats. In this paper, we conduct the first comprehensive survey of the state-of-the-art in privacy attacks and defenses in VFL. We provide taxonomies for both attacks and defenses, based on their characterizations, and discuss open challenges and future research directions. Specifically, our discussion is structured around the model's life cycle, by delving into the privacy threats encountered during different stages of machine learning and their corresponding countermeasures. This survey not only serves as a resource for the research community but also offers clear guidance and actionable insights for practitioners to safeguard data privacy throughout the model's life cycle.
Humans Beat Deep Networks at Recognizing Objects in Unusual Poses, Given Enough Time

Deep learning is closing the gap with humans on several object recognition benchmarks. Here we investigate this gap in the context of challenging images where objects are seen from unusual viewpoints. We find that humans excel at recognizing objects in unusual poses, in contrast with state-of-the-art pretrained networks (EfficientNet, SWAG, ViT, SWIN, BEiT, ConvNext) which are systematically brittle in this condition. Remarkably, as we limit image exposure time, human performance degrades to the level of deep networks, suggesting that additional mental processes (requiring additional time) take place when humans identify objects in unusual poses. Finally, our analysis of error patterns of humans vs. networks reveals that even time-limited humans are dissimilar to feed-forward deep networks. We conclude that more work is needed to bring computer vision systems to the level of robustness of the human visual system. Understanding the nature of the mental processes taking place during extra viewing time may be key to attain such robustness.
A quantitative assessment of Geant4 for predicting the yield and distribution of positron-emitting fragments in ion beam therapy

Purpose: To compare the accuracy with which different hadronic inelastic physics models across ten Geant4 Monte Carlo simulation toolkit versions can predict positron-emitting fragments produced along the beam path during carbon and oxygen ion therapy. Materials and Methods: Phantoms of polyethylene, gelatin or poly(methyl methacrylate) were irradiated with monoenergetic carbon and oxygen ion beams. Post-irradiation, 4D PET images were acquired and parent $^{11}$C, $^{10}$C and $^{15}$O radionuclides contributions in each voxel were determined from the extracted time activity curves. Experiments were simulated in Geant4 Monte Carlo versions 10-11.1, with three different fragmentation models: binary ion cascade (BIC), quantum molecular dynamics (QMD) and the Liege intranuclear cascade (INCL++) - 30 combinations. Total/parent isotope positron annihilation yields were compared between simulations and experiments using normalised mean squared error and Pearson cross-correlation coefficient. Depth of maximum/distal 50\% peak position yield were also compared. Results: Performance varied considerably across versions and models, with no one best predicting all positron-emitting fragments. BIC in Geant4 10.2 provided the best overall agreement with experimental results in the largest number of test cases. QMD consistently provided the best estimates of both the depth of peak positron yield (10.4 and 10.6) and the distal 50\%-of-peak point (10.2), while BIC also performed well and INCL generally performed the worst across most Geant4 versions. Conclusions: Best spatial prediction of annihilation yield and positron-emitting fragment production during carbon and oxygen ion therapy was found to be 10.2. p03 with BIC or QMD. These version/model combinations are recommended for future heavy ion therapy research.
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods. To address this issue, we introduce HarmBench, a standardized evaluation framework for automated red teaming. We identify several desirable properties previously unaccounted for in red teaming evaluations and systematically design HarmBench to meet these criteria. Using HarmBench, we conduct a large-scale comparison of 18 red teaming methods and 33 target LLMs and defenses, yielding novel insights. We also introduce a highly efficient adversarial training method that greatly enhances LLM robustness across a wide range of attacks, demonstrating how HarmBench enables codevelopment of attacks and defenses. We open source HarmBench at https://github.com/centerforaisafety/HarmBench.
Symbol Correctness in Deep Neural Networks Containing Symbolic Layers

To handle AI tasks that combine perception and logical reasoning, recent work introduces Neurosymbolic Deep Neural Networks (NS-DNNs), which contain -- in addition to traditional neural layers -- symbolic layers: symbolic expressions (e.g., SAT formulas, logic programs) that are evaluated by symbolic solvers during inference. We identify and formalize an intuitive, high-level principle that can guide the design and analysis of NS-DNNs: symbol correctness, the correctness of the intermediate symbols predicted by the neural layers with respect to a (generally unknown) ground-truth symbolic representation of the input data. We demonstrate that symbol correctness is a necessary property for NS-DNN explainability and transfer learning (despite being in general impossible to train for). Moreover, we show that the framework of symbol correctness provides a precise way to reason and communicate about model behavior at neural-symbolic boundaries, and gives insight into the fundamental tradeoffs faced by NS-DNN training algorithms. In doing so, we both identify significant points of ambiguity in prior work, and provide a framework to support further NS-DNN developments.
Reducing the Cost of Quantum Chemical Data By Backpropagating Through Density Functional Theory

Density Functional Theory (DFT) accurately predicts the quantum chemical properties of molecules, but scales as $O(N_{\text{electrons}}^3)$. Sch\"utt et al. (2019) successfully approximate DFT 1000x faster with Neural Networks (NN). Arguably, the biggest problem one faces when scaling to larger molecules is the cost of DFT labels. For example, it took years to create the PCQ dataset (Nakata&Shimazaki, 2017) on which subsequent NNs are trained within a week. DFT labels molecules by minimizing energy $E(\cdot )$ as a"loss function. "We bypass dataset creation by directly training NNs with $E(\cdot )$ as a loss function. For comparison, Sch\"utt et al. (2019) spent 626 hours creating a dataset on which they trained their NN for 160h, for a total of 786h; our method achieves comparable performance within 31h.
Generative Modeling of Graphs via Joint Diffusion of Node and Edge Attributes

Graph generation is integral to various engineering and scientific disciplines. Nevertheless, existing methodologies tend to overlook the generation of edge attributes. However, we identify critical applications where edge attributes are essential, making prior methods potentially unsuitable in such contexts. Moreover, while trivial adaptations are available, empirical investigations reveal their limited efficacy as they do not properly model the interplay among graph components. To address this, we propose a joint score-based model of nodes and edges for graph generation that considers all graph components. Our approach offers two key novelties: (i) node and edge attributes are combined in an attention module that generates samples based on the two ingredients; and (ii) node, edge and adjacency information are mutually dependent during the graph diffusion process. We evaluate our method on challenging benchmarks involving real-world and synthetic datasets in which edge features are crucial. Additionally, we introduce a new synthetic dataset that incorporates edge values. Furthermore, we propose a novel application that greatly benefits from the method due to its nature: the generation of traffic scenes represented as graphs. Our method outperforms other graph generation methods, demonstrating a significant advantage in edge-related measures.
DistiLLM: Towards Streamlined Distillation for Large Language Models

Knowledge distillation (KD) is widely used for compressing a teacher model to a smaller student model, reducing its inference cost and memory footprint while preserving model capabilities. However, current KD methods for auto-regressive sequence models (e.g., large language models) suffer from missing a standardized objective function. Moreover, the recent use of student-generated outputs to address training-inference mismatches has significantly escalated computational costs. To tackle these issues, we introduce DistiLLM, a more effective and efficient KD framework for auto-regressive language models. DistiLLM comprises two components: (1) a novel skew Kullback-Leibler divergence loss, where we unveil and leverage its theoretical properties, and (2) an adaptive off-policy approach designed to enhance the efficiency in utilizing student-generated outputs. Extensive experiments, including instruction-following tasks, demonstrate the effectiveness of DistiLLM in building high-performing student models while achieving up to 4.3$\times$ speedup compared to recent KD methods.
Efficient Solvers for Partial Gromov-Wasserstein

The partial Gromov-Wasserstein (PGW) problem facilitates the comparison of measures with unequal masses residing in potentially distinct metric spaces, thereby enabling unbalanced and partial matching across these spaces. In this paper, we demonstrate that the PGW problem can be transformed into a variant of the Gromov-Wasserstein problem, akin to the conversion of the partial optimal transport problem into an optimal transport problem. This transformation leads to two new solvers, mathematically and computationally equivalent, based on the Frank-Wolfe algorithm, that provide efficient solutions to the PGW problem. We further establish that the PGW problem constitutes a metric for metric measure spaces. Finally, we validate the effectiveness of our proposed solvers in terms of computation time and performance on shape-matching and positive-unlabeled learning problems, comparing them against existing baselines.
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection

Knowledge hallucination have raised widespread concerns for the security and reliability of deployed LLMs. Previous efforts in detecting hallucinations have been employed at logit-level uncertainty estimation or language-level self-consistency evaluation, where the semantic information is inevitably lost during the token-decoding procedure. Thus, we propose to explore the dense semantic information retained within LLMs' \textbf{IN}ternal \textbf{S}tates for halluc\textbf{I}nation \textbf{DE}tection (\textbf{INSIDE}). In particular, a simple yet effective \textbf{EigenScore} metric is proposed to better evaluate responses' self-consistency, which exploits the eigenvalues of responses' covariance matrix to measure the semantic consistency/diversity in the dense embedding space. Furthermore, from the perspective of self-consistent hallucination detection, a test time feature clipping approach is explored to truncate extreme activations in the internal states, which reduces overconfident generations and potentially benefits the detection of overconfident hallucinations. Extensive experiments and ablation studies are performed on several popular LLMs and question-answering (QA) benchmarks, showing the effectiveness of our proposal.
SRF Cavity as Galactic Dark Photon Telescope

Dark photons, aside from constituting non-relativistic dark matter, can also be generated relativistically through the decay or annihilation of other dark matter candidates, contributing to a galactic dark photon background. The production of dark photons tends to favor specific polarization modes, determined by the microscopic coupling between dark matter and dark photons. We leverage data obtained from previous searches for dark photon dark matter using a superconducting radio-frequency cavity to explore galactic dark photon fluxes. The interplay of anisotropic directions and Earth's rotation introduces a diurnal modulation of signals within the cavities, manifesting distinct variation patterns for longitudinal and transverse modes. Our findings highlight the efficacy of superconducting radio-frequency cavities, characterized by significantly high-quality factors, as powerful telescopes for detecting galactic dark photons, unveiling a novel avenue in the indirect search for dark matter through multi-messenger astronomy.
Bringing together two paradigms of non-equilibrium: Driven dynamics of aging systems

There are two fundamental paradigms for non-equilibrium dynamics: on the one hand, aging towards an equilibrium state that cannot be reached on reasonable timescales; on the other, external driving that can lead to non-equilibrium steady states. We explore how these two mechanisms interact, by studying the behaviour of trap models, which are paradigmatic descriptions of slow glassy dynamics, when driven by trajectory bias towards high or low activity. To diagnose whether the driven systems continue to age, we establish a framework for mapping the biased dynamics to a Markovian time evolution with time-dependent transition rates. We find that the original aging dynamics reacts in two qualitatively distinct ways to the driving: it can be destroyed by driving of any nonzero strength (fragile aging), whereby the dynamics either reaches an active steady state or effectively freezes; or it can persist within a finite range of driving strengths around the undriven case (robust aging). This classification into fragile and robust aging could form the basis for distinguishing different universality classes of aging dynamics.
FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuning

Modern recommender systems may output considerably different recommendations due to small perturbations in the training data. Changes in the data from a single user will alter the recommendations as well as the recommendations of other users. In applications like healthcare, housing, and finance, this sensitivity can have adverse effects on user experience. We propose a method to stabilize a given recommender system against such perturbations. This is a challenging task due to (1) the lack of a ``reference'' rank list that can be used to anchor the outputs; and (2) the computational challenges in ensuring the stability of rank lists with respect to all possible perturbations of training data. Our method, FINEST, overcomes these challenges by obtaining reference rank lists from a given recommendation model and then fine-tuning the model under simulated perturbation scenarios with rank-preserving regularization on sampled items. Our experiments on real-world datasets demonstrate that FINEST can ensure that recommender models output stable recommendations under a wide range of different perturbations without compromising next-item prediction accuracy.