Miracle-dz/UniFAR-FTUs huggingface dataset:
Graph-based Time-Series Anomaly Detection: A Survey

With the recent advances in technology, a wide range of systems continue to collect a large amount of data over time and thus generate time series. Time-Series Anomaly Detection (TSAD) is an important task in various time-series applications such as e-commerce, cybersecurity, vehicle maintenance, and healthcare monitoring. However, this task is very challenging as it requires considering both the intra-variable dependency and the inter-variable dependency, where a variable can be defined as an observation in time series data. Recent graph-based approaches have made impressive progress in tackling the challenges of this field. In this survey, we conduct a comprehensive and up-to-date review of Graph-based TSAD (G-TSAD). First, we explore the significant potential of graph representation learning for time-series data. Then, we review state-of-the-art graph anomaly detection techniques in the context of time series and discuss their strengths and drawbacks. Finally, we discuss the technical challenges and potential future directions for possible improvements in this research field.
Hierarchical Classification of Research Fields in the "Web of Science" Using Deep Learning

This paper presents a hierarchical classification system that automatically categorizes a scholarly publication using its abstract into a three-tier hierarchical label set (discipline, field, subfield) in a multi-class setting. This system enables a holistic categorization of research activities in the mentioned hierarchy in terms of knowledge production through articles and impact through citations, permitting those activities to fall into multiple categories. The classification system distinguishes 44 disciplines, 718 fields and 1,485 subfields among 160 million abstract snippets in Microsoft Academic Graph (version 2018-05-17). We used batch training in a modularized and distributed fashion to address and allow for interdisciplinary and interfield classifications in single-label and multi-label settings. In total, we have conducted 3,140 experiments in all considered models (Convolutional Neural Networks, Recurrent Neural Networks, Transformers). The classification accuracy is>90% in 77.13% and 78.19% of the single-label and multi-label classifications, respectively. We examine the advantages of our classification by its ability to better align research texts and output with disciplines, to adequately classify them in an automated way, and to capture the degree of interdisciplinarity. The proposed system (a set of pre-trained models) can serve as a backbone to an interactive system for indexing scientific publications in the future.
Optimal Learning of Deep Random Networks of Extensive-width

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width are proportionally large. We propose a closed-form expression for the Bayes-optimal test error, for regression and classification tasks. We further compute closed-form expressions for the test errors of ridge regression, kernel and random features regression. We find, in particular, that optimally regularized ridge regression, as well as kernel regression, achieve Bayes-optimal performances, while the logistic loss yields a near-optimal test error for classification. We further show numerically that when the number of samples grows faster than the dimension, ridge and kernel methods become suboptimal, while neural networks achieve test error close to zero from quadratically many samples.
NASiam: Efficient Representation Learning using Neural Architecture Search for Siamese Networks

Siamese networks are one of the most trending methods to achieve self-supervised visual representation learning (SSL). Since hand labeling is costly, SSL can play a crucial part by allowing deep learning to train on large unlabeled datasets. Meanwhile, Neural Architecture Search (NAS) is becoming increasingly important as a technique to discover novel deep learning architectures. However, early NAS methods based on reinforcement learning or evolutionary algorithms suffered from ludicrous computational and memory costs. In contrast, differentiable NAS, a gradient-based approach, has the advantage of being much more efficient and has thus retained most of the attention in the past few years. In this article, we present NASiam, a novel approach that uses for the first time differentiable NAS to improve the multilayer perceptron projector and predictor (encoder/predictor pair) architectures inside siamese-networks-based contrastive learning frameworks (e.g., SimCLR, SimSiam, and MoCo) while preserving the simplicity of previous baselines. We crafted a search space designed explicitly for multilayer perceptrons, inside which we explored several alternatives to the standard ReLU activation function. We show that these new architectures allow ResNet backbone convolutional models to learn strong representations efficiently. NASiam reaches competitive performance in both small-scale (i.e., CIFAR-10/CIFAR-100) and large-scale (i.e., ImageNet) image classification datasets while costing only a few GPU hours. We discuss the composition of the NAS-discovered architectures and emit hypotheses on why they manage to prevent collapsing behavior. Our code is available at https://github.com/aheuillet/NASiam.
Dynamic Flows on Curved Space Generated by Labeled Data

The scarcity of labeled data is a long-standing challenge for many machine learning tasks. We propose our gradient flow method to leverage the existing dataset (i.e., source) to generate new samples that are close to the dataset of interest (i.e., target). We lift both datasets to the space of probability distributions on the feature-Gaussian manifold, and then develop a gradient flow method that minimizes the maximum mean discrepancy loss. To perform the gradient flow of distributions on the curved feature-Gaussian space, we unravel the Riemannian structure of the space and compute explicitly the Riemannian gradient of the loss function induced by the optimal transport metric. For practical applications, we also propose a discretized flow, and provide conditional results guaranteeing the global convergence of the flow to the optimum. We illustrate the results of our proposed gradient flow method on several real-world datasets and show our method can improve the accuracy of classification models in transfer learning settings.
Effectiveness of Moving Target Defenses for Adversarial Attacks in ML-based Malware Detection

Several moving target defenses (MTDs) to counter adversarial ML attacks have been proposed in recent years. MTDs claim to increase the difficulty for the attacker in conducting attacks by regularly changing certain elements of the defense, such as cycling through configurations. To examine these claims, we study for the first time the effectiveness of several recent MTDs for adversarial ML attacks applied to the malware detection domain. Under different threat models, we show that transferability and query attack strategies can achieve high levels of evasion against these defenses through existing and novel attack strategies across Android and Windows. We also show that fingerprinting and reconnaissance are possible and demonstrate how attackers may obtain critical defense hyperparameters as well as information about how predictions are produced. Based on our findings, we present key recommendations for future work on the development of effective MTDs for adversarial attacks in ML-based malware detection.
Weight Prediction Boosts the Convergence of AdamW

In this paper, we introduce weight prediction into the AdamW optimizer to boost its convergence when training the deep neural network (DNN) models. In particular, ahead of each mini-batch training, we predict the future weights according to the update rule of AdamW and then apply the predicted future weights to do both forward pass and backward propagation. In this way, the AdamW optimizer always utilizes the gradients w.r. t. the future weights instead of current weights to update the DNN parameters, making the AdamW optimizer achieve better convergence. Our proposal is simple and straightforward to implement but effective in boosting the convergence of DNN training. We performed extensive experimental evaluations on image classification and language modeling tasks to verify the effectiveness of our proposal. The experimental results validate that our proposal can boost the convergence of AdamW and achieve better accuracy than AdamW when training the DNN models.
Transient Stellar Collisions as Multimessenger Probes: Nonthermal, Gravitational-wave Emission and the Cosmic Ladder Argument

In dense stellar clusters like galactic nuclei and globular clusters, stellar densities are so high that stars might physically collide with each other. In galactic nuclei the energy and power output can be close to, and even exceed, those from supernovae events. We address the event rate and the electromagnetic characteristics of collisions of main-sequence stars (MS) and red giants (RGs). We also investigate the case in which the cores form a binary and emit gravitational waves. In the case of RGs, this is particularly interesting because the cores are degenerate. We find that MS event rate can be as high as tens per year, and that of RGs 1 order of magnitude larger. The collisions are powerful enough to mimic supernovae or tidal disruptions events. We find Zwicky Transient Facility observational data that seem to exhibit the features we describe. The cores embedded in the gaseous debris experience a friction force that has an impact on the chirping mass of the gravitational wave. As a consequence, the two small cores in principle mimic two supermassive black holes merging. However, their evolution in frequency along with the precedent electromagnetic burst and the ulterior afterglow are efficient tools to reveal the impostors. In the particular case of RGs, we derive the properties of the degenerate He cores and their H-burning shells to analyze the formation of the binaries. The merger is such that it can be misclassified with SN Ia events. Because the masses and densities of the cores are so dissimilar in values depending on their evolutionary stage, the argument about standard candles and cosmic ladder should be reevaluated.
The geometry of hidden representations of large transformer models

Large transformers are powerful architectures used for self-supervised data analysis across various data types, including protein sequences, images, and text. In these models, the semantic structure of the dataset emerges from a sequence of transformations between one representation and the next. We characterize the geometric and statistical properties of these representations and how they change as we move through the layers. By analyzing the intrinsic dimension (ID) and neighbor composition, we find that the representations evolve similarly in transformers trained on protein language tasks and image reconstruction tasks. In the first layers, the data manifold expands, becoming high-dimensional, and then contracts significantly in the intermediate layers. In the last part of the model, the ID remains approximately constant or forms a second shallow peak. We show that the semantic information of the dataset is better expressed at the end of the first peak, and this phenomenon can be observed across many models trained on diverse datasets. Based on our findings, we point out an explicit strategy to identify, without supervision, the layers that maximize semantic content: representations at intermediate layers corresponding to a relative minimum of the ID profile are more suitable for downstream learning tasks.
LS-category and topological complexity of several families of fibre bundles

In this paper, we study upper bounds for the topological complexity of the total spaces of some classes of fibre bundles. We calculate a tight upper bound for the topological complexity of an $n$-dimensional Klein bottle. We also compute the exact value of the topological complexity of $3$-dimensional Klein bottle. We describe the cohomology rings of several classes of generalized projective product spaces with $\mathbb{Z}_2$-coefficients. Then we study the LS-category and topological complexity of infinite families of generalized projective product spaces. We reckon the exact value of these invariants in many specific cases. We calculate the equivariant LS-category and equivariant topological complexity of several product spaces equipped with $\mathbb{Z}_2$-action.
Detecting Histologic Glioblastoma Regions of Prognostic Relevance

Glioblastoma is the most common and aggressive malignant adult tumor of the central nervous system, with grim prognosis and heterogeneous morphologic and molecular proﬁles. Since the adoption of the current standard of care treatment, 18 years ago, there are no substantial prognostic improvements noticed. Accurate prediction of patient overall survival (OS) from clinical histopathology whole slide images (WSI) us-ing advanced computational methods could contribute to optimization of clinical decision making and patient management. Here, we focus on identifying prognostically relevant glioblastoma morphologic patterns on H&E stained WSI. The exact approach capitalizes on the comprehensive WSI curation of apparent artifactual content and on an interpretability mechanism via a weakly supervised attention based multiple instance learning algorithm that further utilizes clustering to constrain the search space. The automatically identiﬁed patterns of high diagnostic value are used to classify the WSI as representative of a short-or a long-survivor. Identifying tumor morphologic patterns associated with short and long OS will allow the clinical neuropathologist to provide additional prognostic information gleaned during microscopic assessment to the treating team, as well as suggest avenues of biological investigation for understanding and potentially treating glioblastoma.
Three-point functions of conserved supercurrents in 3D 
N=1
 SCFT: General formalism for arbitrary superspins

We analyse the general structure of the three-point functions of conserved higher-spin supercurrents in 3D, $\mathcal{N}=1$ superconformal field theory. It is shown that supersymmetry imposes additional restrictions on correlation functions of conserved higher-spin currents. We develop a manifestly supersymmetric formalism to compute the three-point function $\langle \mathbf{J}^{}_{s_{1}} \mathbf{J}'_{s_{2}} \mathbf{J}''_{s_{3}} \rangle$, where $\mathbf{J}^{}_{s_{1}}$, $\mathbf{J}'_{s_{2}}$ and $\mathbf{J}''_{s_{3}}$ are conserved higher-spin supercurrents with superspins $s_{1}$, $s_{2}$ and $s_{3}$ respectively (integer or half-integer). Using a computational approach limited only by computer power, we analytically impose the constraints arising from the superfield conservation equations and symmetries under permutations of superspace points. Explicit solutions for three-point functions are presented and we provide a complete classification of the results for $s_{i} \leq 20 $; the pattern is very clear, and we propose that our classification holds for arbitrary superspins. We demonstrate that Grassmann-even three-point functions are fixed up to one parity-even structure and one parity-odd structure, while Grassmann-odd three-point functions are fixed up to a single parity-even structure. The existence of the parity-odd structure in the Grassmann-even correlation functions is subject to a set of triangle inequalities in the superspins. For completeness, we also analyse the structure of three-point functions involving conserved higher-spin supercurrents and scalar superfields.
A Survey of Methods, Challenges and Perspectives in Causality

Deep Learning models have shown success in a large variety of tasks by extracting correlation patterns from high-dimensional data but still struggle when generalizing out of their initial distribution. As causal engines aim to learn mechanisms independent from a data distribution, combining Deep Learning with Causality can have a great impact on the two fields. In this paper, we further motivate this assumption. We perform an extensive overview of the theories and methods for Causality from different perspectives, with an emphasis on Deep Learning and the challenges met by the two domains. We show early attempts to bring the fields together and the possible perspectives for the future. We finish by providing a large variety of applications for techniques from Causality.
Differentially-Private Hierarchical Clustering with Provable Approximation Guarantees

Hierarchical Clustering is a popular unsupervised machine learning method with decades of history and numerous applications. We initiate the study of differentially private approximation algorithms for hierarchical clustering under the rigorous framework introduced by (Dasgupta, 2016). We show strong lower bounds for the problem: that any $\epsilon$-DP algorithm must exhibit $O(|V|^2/ \epsilon)$-additive error for an input dataset $V$. Then, we exhibit a polynomial-time approximation algorithm with $O(|V|^{2.5}/ \epsilon)$-additive error, and an exponential-time algorithm that meets the lower bound. To overcome the lower bound, we focus on the stochastic block model, a popular model of graphs, and, with a separation assumption on the blocks, propose a private $1+o(1)$ approximation algorithm which also recovers the blocks exactly. Finally, we perform an empirical study of our algorithms and validate their performance.
Exploring Wedges of an Oriented Grid by an Automaton with Pebbles

A mobile agent, modeled as a deterministic finite automaton, navigates in the infinite anonymous oriented grid $\mathbb{Z} \times \mathbb{Z}$. It has to explore a given infinite subgraph of the grid by visiting all of its nodes. We focus on the simplest subgraphs, called {\em wedges}, spanned by all nodes of the grid located between two half-lines in the plane, with a common origin. Many wedges turn out to be impossible to explore by an automaton that cannot mark nodes of the grid. Hence, we study the following question: Given a wedge $W$, what is the smallest number $p$ of (movable) pebbles for which there exists an automaton that can explore $W$ using $p$ pebbles? Our main contribution is a complete solution of this problem. For each wedge $W$ we determine this minimum number $p$, show an automaton that explores it using $p$ pebbles and show that fewer pebbles are not enough. We show that this smallest number of pebbles can vary from 0 to 3, depending on the angle between half-lines limiting the wedge and depending on whether the automaton can cross these half-lines or not.
Filtering Context Mitigates Scarcity and Selection Bias in Political Ideology Prediction

We propose a novel supervised learning approach for political ideology prediction (PIP) that is capable of predicting out-of-distribution inputs. This problem is motivated by the fact that manual data-labeling is expensive, while self-reported labels are often scarce and exhibit significant selection bias. We propose a novel statistical model that decomposes the document embeddings into a linear superposition of two vectors; a latent neutral \emph{context} vector independent of ideology, and a latent \emph{position} vector aligned with ideology. We train an end-to-end model that has intermediate contextual and positional vectors as outputs. At deployment time, our model predicts labels for input documents by exclusively leveraging the predicted positional vectors. On two benchmark datasets we show that our model is capable of outputting predictions even when trained with as little as 5\% biased data, and is significantly more accurate than the state-of-the-art. Through crowd-sourcing we validate the neutrality of contextual vectors, and show that context filtering results in ideological concentration, allowing for prediction on out-of-distribution examples.
Discovery of a Spatially and Kinematically Resolved 55 kpc Scale Superbubble Inflated by an Intermediate-redshift Non-BAL Quasar

We report on the discovery of a rare case of spatially and kinematically resolved galactic-scale outflow at intermediate redshift based on VLT/MUSE optical integral field spectroscopic observation of the quasar HE 0238–1904. This classical non–broad absorption line quasar at z = 0.631 remains underexplored in its optical emission lines, though its UV absorption lines are well studied. We identify a superbubble driven by HE 0238−1904 from the emission line morphology, line ratio diagnostics, and kinematics showing a one-sided outflow reaching a projected distance of R ∼ 55 kpc from the nucleus. The bulk of the ionized gas, with a characteristic mass M ∼ 108 M ⊙, is blueshifted by v ≈ 700 km s−1 with respect to the quasar systemic velocity. The outflows detected using the absorption and emission lines are likely stratified components of different spatial scale and velocity in the ionized phase outflow. Although feedback in HE 0238–1904 is taking place on kiloparsec scales, the kinetic power of the outflow at 55 kpc (≪0.1% L bol) implies that it is inadequate to effectively regulate the evolution of the host galaxy at this large scale.
Stable Target Field for Reduced Variance Score Estimation in Diffusion Models

Diffusion models generate samples by reversing a fixed forward diffusion process. Despite already providing impressive empirical results, these diffusion models algorithms can be further improved by reducing the variance of the training targets in their denoising score-matching objective. We argue that the source of such variance lies in the handling of intermediate noise-variance scales, where multiple modes in the data affect the direction of reverse paths. We propose to remedy the problem by incorporating a reference batch which we use to calculate weighted conditional scores as more stable training targets. We show that the procedure indeed helps in the challenging intermediate regime by reducing (the trace of) the covariance of training targets. The new stable targets can be seen as trading bias for reduced variance, where the bias vanishes with increasing reference batch size. Empirically, we show that the new objective improves the image quality, stability, and training speed of various popular diffusion models across datasets with both general ODE and SDE solvers. When used in combination with EDM, our method yields a current SOTA FID of 1.90 with 35 network evaluations on the unconditional CIFAR-10 generation task. The code is available at https://github.com/Newbeeer/stf
Simultaneous Estimation of Overall and Domain Abilities: A Higher-Order IRT Model Approach

Assessments consisting of different domains (e.g., content areas, objectives) are typically multidimensional in nature but are commonly assumed to be unidimensional for estimation purposes. The different domains of these assessments are further treated as multi-unidimensional tests for the purpose of obtaining diagnostic information. However, when the domains are disparate, assuming a single underlying ability across the domains is not tenable. Moreover, estimating domain proficiencies based on short tests can result in unreliable scores. This article presents a higher-order item response theory framework where an overall and multiple domain abilities are specified in the same model. Using a Markov chain Monte Carlo method in a hierarchical Bayesian framework, the overall and domain-specific abilities, and their correlations, are estimated simultaneously. The feasibility and effectiveness of the proposed model are investigated under varied conditions in a simulation study and illustrated using actual assessment data. Implications of the model for future test analysis and ability estimation are also discussed. Index terms: higher-order ability estimation, item response theory, multidimensionality, domain scoring, Markov chain Monte Carlo