Miracle-dz/UniFAR-FTUs huggingface dataset:
Elementary proofs of four standard results on false discovery rate

We collect self-contained elementary proofs of four standard results in the literature on the false discovery rate of the Benjamini-Hochberg (BH) procedure for independent or positive-regression dependent p-values, the Benjamini-Yekutieli procedure for arbitrarily dependent p-values, and the e-BH procedure for arbitrarily dependent e-values. As a corollary, the above proofs also lead to some inequalities of Simes and Hommel. 1 False discovery rate This short note contains proofs of four standard results on the false discovery rate (FDR) of the Benjamini-Hochberg (BH), Benjamini-Yekutieli (BY) and e-BH procedures proposed by, respectively, Benjamini and Hochberg (1995), Benjamini and Yekutieli (2001) and Wang and Ramdas (2022). All results are known and various proofs exist in the literature. Our proofs are elementary, concise, and self-contained. They may not be useful for experts in the field, but they may become helpful for someone who is trained in probability theory but not much in multiple hypothesis testing (e.g., a graduate student). This note serves pedagogical purposes and does not contain any new results. We first briefly define false discovery rate in multiple hypothesis testing. Let H1, . , HK be K hypotheses, and denote by K = {1, . ,K}. The true datagenerating probability measure is denoted by P. For each k ∈ K, Hk is called a true null hypothesis if P ∈ Hk, and the set N ⊆ K be the set of indices of true null hypotheses, which is unknown to the decision maker. Let K0 = |N | be the number of true null hypotheses. A p-variable P is a random variable that satisfies P(P ≤ α) ≤ α for all α ∈ (0, 1). For each k ∈ K, Hk is associated with p-value pk, which is a realization of a random variable Pk being a p-variable if k ∈ N . For convenience, we will use “p-value” for both the random variable Pk and its realized value. ∗Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ontario, Canada. E-mail: wang@uwaterloo.ca.
Optimal transport for causal discovery

To determine causal relationships between two variables, approaches based on Functional Causal Models (FCMs) have been proposed by properly restricting model classes; however, the performance is sensitive to the model assumptions, which makes it difficult to use. In this paper, we provide a novel dynamical-system view of FCMs and propose a new framework for identifying causal direction in the bivariate case. We first show the connection between FCMs and optimal transport, and then study optimal transport under the constraints of FCMs. Furthermore, by exploiting the dynamical interpretation of optimal transport under the FCM constraints, we determine the corresponding underlying dynamical process of the static cause-effect pair data. It provides a new dimension for describing static causal discovery tasks while enjoying more freedom for modeling the quantitative causal influences. In particular, we show that Additive Noise Models (ANMs) correspond to volume-preserving pressureless flows. Consequently, based on their velocity field divergence, we introduce a criterion for determining causal direction. With this criterion, we propose a novel optimal transport-based algorithm for ANMs which is robust to the choice of models and extend it to post-nonlinear models. Our method demonstrated state-of-the-art results on both synthetic and causal discovery benchmark datasets.
Neural Architecture Searching for Facial Attributes-based Depression Recognition

Recent studies show that depression can be partially reflected from human facial attributes. Since facial attributes have various data structures and carry different information, existing approaches fail to specifically consider the optimal way to extract depression-related features from each of them, as well as investigate the best fusion strategy. In this paper, we propose to extend Neural Architecture Search (NAS) technique to design an optimal model for multiple facial attributes-based depression recognition, which can be efficiently and robustly implemented in a small dataset. Our approach first conducts a warmer up step to the feature extractor of each facial attribute, aiming to largely reduce the search space and provide customized architecture, where each feature extractor can be either a Convolution Neural Networks (CNN) or Graph Neural Networks (GNN). Then, we conduct an end-to-end architecture search for all feature extractors and the fusion network, allowing the complementary depression cues to be optimally combined with less redundancy. The experimental results on AVEC 2016 dataset show that the model explored by our approach achieves breakthrough performance with 27% and 30% RMSE and MAE improvements over the existing state-of-the-art. In light of these findings, this paper provides solid evidence and a strong baseline for applying NAS to time-series data-based mental health analysis.
Variational Autoencoder-Based Metamodeling for Multi-Objective Topology Optimization of Electrical Machines

Conventional magneto-static finite element (FE) analysis of electrical machine design is time-consuming and computationally expensive. Since each machine topology has a distinct set of parameters, design optimization is commonly performed independently. This article presents a novel method for predicting key performance indicators (KPIs) of differently parameterized electrical machine topologies at the same time by mapping a high-dimensional integrated design parameters in a lower-dimensional latent space using a variational autoencoder (VAE). After training, via a latent space, the decoder and multi-layer neural network will function as meta-models for sampling new designs and predicting associated KPIs, respectively. This enables parameter-based concurrent multi-topology optimization.
SEN12MS-CR-TS: A Remote Sensing Data Set for Multi-modal Multi-temporal Cloud Removal

About half of all optical observations collected via spaceborne satellites are affected by haze or clouds. Consequently, cloud coverage affects the remote sensing practitioner’s capabilities of a continuous and seamless monitoring of our planet. This work addresses the challenge of optical satellite image reconstruction and cloud removal by proposing a novel multi-modal and multi-temporal data set called SEN12MS-CR-TS. We propose two models highlighting the benefits and use cases of SEN12MS-CR-TS: First, a multi-modal multi-temporal 3D-Convolution Neural Network that predicts a cloud-free image from a sequence of cloudy optical and radar images. Second, a sequence-to-sequence translation model that predicts a cloud-free time series from a cloud-covered time series. Both approaches are evaluated experimentally, with their respective models trained and tested on SEN12MS-CR-TS. The conducted experiments highlight the contribution of our data set to the remote sensing community as well as the benefits of multi-modal and multi-temporal information to reconstruct noisy information. Our data set is available at https://patrickTUM.github. io/cloud removal.
The Time Domain Spectroscopic Survey: Changing-look Quasar Candidates from Multi-epoch Spectroscopy in SDSS-IV

Active galactic nuclei (AGN) can vary significantly in their rest-frame optical/UV continuum emission, and with strong associated changes in broad line emission, on much shorter timescales than predicted by standard models of accretion disks around supermassive black holes. Most such changing-look or changing-state AGN—and at higher luminosities, changing-look quasars (CLQs)—have been found via spectroscopic follow-up of known quasars showing strong photometric variability. The Time Domain Spectroscopic Survey of the Sloan Digital Sky Survey IV (SDSS-IV) includes repeat spectroscopy of large numbers of previously known quasars, many selected irrespective of photometric variability, and with spectral epochs separated by months to decades. Our visual examination of these repeat spectra for strong broad line variability yielded 61 newly discovered CLQ candidates. We quantitatively compare spectral epochs to measure changes in continuum and Hβ broad line emission, finding 19 CLQs, of which 15 are newly recognized. The parent sample includes only broad line quasars, so our study tends to find objects that have dimmed, i.e., turn-off CLQs. However, we nevertheless find four turn-on CLQs that meet our criteria, albeit with broad lines in both dim and bright states. We study the response of Hβ and Mg ii emission lines to continuum changes. The Eddington ratios of CLQs are low, and/or their Hβ broad line width is large relative to the overall quasar population. Repeat quasar spectroscopy in the upcoming SDSS-V black hole Mapper program will reveal significant numbers of CLQs, enhancing our understanding of the frequency and duty cycle of such strong variability, and the physics and dynamics of the phenomenon.
Examining AGN UV/Optical Variability beyond the Simple Damped Random Walk

We present damped harmonic oscillator (DHO) light-curve modeling for a sample of 12,714 spectroscopically confirmed quasars in the Sloan Digital Sky Survey Stripe 82 region. DHO is a second-order continuous-time autoregressive moving-average process, which can be fully described using four independent parameters: a natural oscillation frequency (ω 0), a damping ratio (ξ), a characteristic perturbation timescale (τ perturb), and an amplitude for the perturbing white noise (σ ϵ ). The asymptotic variability amplitude of a DHO process is quantified by σ DHO—a function of ω 0, ξ, τ perturb, and σ ϵ . We find that both τ perturb and σ ϵ follow different dependencies with rest-frame wavelength (λ RF) on either side of 2500 Å, whereas σ DHO follows a single power-law relation with λ RF. After correcting for wavelength dependence, σ DHO exhibits anticorrelations with both the Eddington ratio and the black hole mass, while τ perturb—with a typical value of days in the rest frame—shows an anticorrelation with the bolometric luminosity. Modeling active galactic nuclei (AGN) variability as a DHO offers more insight into the workings of accretion disks close to the supermassive black holes at the center of AGN. The newly discovered short-term variability (characterized by τ perturb and σ ϵ ) and its correlation with bolometric luminosity pave the way for new algorithms that will derive fundamental properties (e.g., Eddington ratio) of AGN using photometric data alone.
Combining mixed effects hidden Markov models with latent alternating recurrent event processes to model diurnal active–rest cycles

Data collected from wearable devices can shed light on an individual's pattern of behavioral and circadian routine. Phone use can be modeled as alternating processes, between the state of active use and the state of being idle. Markov chains and alternating recurrent event models are commonly used to model state transitions in cases such as these, and the incorporation of random effects can be used to introduce diurnal effects. While state labels can be derived prior to modeling dynamics, this approach omits informative regression covariates that can influence state memberships. We instead propose an alternating recurrent event proportional hazards (PH) regression to model the transitions between latent states. We propose an expectation–maximization algorithm for imputing latent state labels and estimating parameters. We show that our E‐step simplifies to the hidden Markov model (HMM) forward–backward algorithm, allowing us to recover an HMM with logistic regression transition probabilities. In addition, we show that PH modeling of discrete‐time transitions implicitly penalizes the logistic regression likelihood and results in shrinkage estimators for the relative risk. This new estimator favors an extended stay in a state and is useful for modeling diurnal rhythms. We derive asymptotic distributions for our parameter estimates and compare our approach against competing methods through simulation as well as in a digital phenotyping study that followed smartphone use in a cohort of adolescents with mood disorders.
Exploration of Hyperdimensional Computing Strategies for Enhanced Learning on Epileptic Seizure Detection

Wearable and unobtrusive monitoring and prediction of epileptic seizures has the potential to significantly increase the life quality of patients, but is still an unreached goal due to challenges of real-time detection and wearable devices design. Hyperdimensional (HD) computing has evolved in recent years as a new promising machine learning approach, especially when talking about wearable applications. But in the case of epilepsy detection, standard HD computing is not performing at the level of other state-of-the-art algorithms. This could be due to the inherent complexity of the seizures and their signatures in different biosignals, such as the electroencephalogram (EEG), the highly personalized nature, and the disbalance of seizure and non-seizure instances. In the literature, different strategies for improved learning of HD computing have been proposed, such as iterative (multi-pass) learning, multi-centroid learning and learning with sample weight (“OnlineHD”). Yet, most of them have not been tested on the challenging task of epileptic seizure detection, and it stays unclear whether they can increase the HD computing performance to the level of the current state-of-the-art algorithms for wearable devices, such as random forests. Thus, in this paper, we implement different learning strategies and assess their performance on an individual basis, or in combination, regarding detection performance and memory and computational requirements. Results show that the best-performing algorithm, which is a combination of multi-centroid and multi-pass, can indeed reach the performance of the random forest model on a highly unbalanced dataset imitating a real-life epileptic seizure detection application.
Learning Graph Augmentations to Learn Graph Representations

Devising augmentations for graph contrastive learning is challenging due to their irregular structure, drastic distribution shifts, and nonequivalent feature spaces across datasets. We introduce LG2AR, Learning Graph Augmentations to Learn Graph Representations, which is an end-to-end automatic graph augmentation framework that helps encoders learn generalizable representations on both node and graph levels. LG2AR consists of a probabilistic policy that learns a distribution over augmentations and a set of probabilistic augmentation heads that learn distributions over augmentation parameters. We show that LG2AR achieves state-of-the-art results on 18 out of 20 graph-level and node-level benchmarks compared to previous unsupervised models under both linear and semi-supervised evaluation protocols. The source code will be released here: https://github.com/kavehhassani/lg2ar
Proximity, remoteness and maximum degree in graphs

The average distance of a vertex $v$ of a connected graph $G$ is the arithmetic mean of the distances from $v$ to all other vertices of $G$. The proximity $\pi(G)$ and the remoteness $\rho(G)$ of $G$ are the minimum and the maximum of the average distances of the vertices of $G$, respectively. In this paper, we give upper bounds on the remoteness and proximity for graphs of given order, minimum degree and maximum degree. Our bounds are sharp apart from an additive constant.
Reinforcing a Philosophy: A counting approach to square functions over local fields

In this paper, we revisit the square function estimates for extension operators over curves with torsion and prove results for planar curves of the form $(T,\phi(T))$ where $\phi(T)$ is a polynomial of degree at least 2. This includes new results for degenerate, finite type curves endowed with the Euclidean arclength measure such as those given by monomials $\phi(T) = T^k$ for $k \geq 3$. Our results are uniform over all local fields of characteristic 0.
Neuronal Correlation: a Central Concept in Neural Network

This paper proposes to study neural networks through neuronal correlation, a statistical measure of correlated neuronal activity on the penultimate layer. We show that neuronal correlation can be efficiently estimated via weight matrix, can be effectively enforced through layer structure, and is a strong indicator of generalisation ability of the network. More importantly, we show that neuronal correlation significantly impacts on the accuracy of entropy estimation in high-dimensional hidden spaces. While previous estimation methods may be subject to significant inaccuracy due to implicit assumption on neuronal independence, we present a novel computational method to have an efficient and authentic computation of entropy, by taking into consideration the neuronal correlation. In doing so, we install neuronal correlation as a central concept of neural network.
Seasonality and spatial dependence of meso- and submesoscale ocean currents from along-track satellite altimetry

Along-track wavenumber spectral densities of sea surface height (SSH) are estimated from Jason-2 altimetry data as a function of spatial location and calendar month, to understand the seasonality of meso- and submesoscale balanced dynamics across the global ocean. Regions with significant mode-1 and mode-2 baroclinic tides are rejected, restricting the analysis to the extratropics. Where balanced motion dominates, the SSH spectral density is averaged over all pass segments in a region for each calendar month, and is fit to a 4-parameter model consisting of a flat plateau at low wavenumbers, a transition at wavenumber k0 to a red power law spectrum k−s, and a white spectrum at high wavenumbers that models the altimeter noise. The monthly time series of the model parameters are compared to the evolution of the mixed layer. The annual mode of the spectral slope s reaches a minimum after the mixed layer deepens, and the annual mode of the bandpassed kinetic energy in the ranges [2k0,4k0] and [k0,2k0] peak ∼2 and ∼4 months, respectively, after the maximum of the annual mode of the mixed layer depth. This analysis is consistent with an energization of the submesoscale by a winter mixed layer instability followed by an inverse cascade of kinetic energy to the mesoscale, in agreement with prior modeling studies and in situ measurements. These results are compared to prior modeling, in situ, and satellite investigations of specific regions, and are broadly consistent with them within measurement uncertainties.
Increasing the Cost of Model Extraction with Calibrated Proof of Work

In model extraction attacks, adversaries can steal a machine learning model exposed via a public API by repeatedly querying it and adjusting their own model based on obtained predictions. To prevent model stealing, existing defenses focus on detecting malicious queries, truncating, or distorting outputs, thus necessarily introducing a tradeoff between robustness and model utility for legitimate users. Instead, we propose to impede model extraction by requiring users to complete a proof-of-work before they can read the model's predictions. This deters attackers by greatly increasing (even up to 100x) the computational effort needed to leverage query access for model extraction. Since we calibrate the effort required to complete the proof-of-work to each query, this only introduces a slight overhead for regular users (up to 2x). To achieve this, our calibration applies tools from differential privacy to measure the information revealed by a query. Our method requires no modification of the victim model and can be applied by machine learning practitioners to guard their publicly exposed models against being easily stolen.
Frequency and Phase Synchronization in Distributed Antenna Arrays Based on Consensus Averaging and Kalman Filtering

A decentralized approach for joint frequency and phase synchronization in distributed antenna arrays is presented. The nodes in the array locally broadcast their frequencies and phases to their neighboring nodes, and use consensus averaging to align these parameters across the array. The architecture is fully distributed, requiring no centralization. Each node has a local oscillator and we consider a signal model where intrinsic frequency and phase errors of the local oscillators on each node caused by the frequency drift and phase jitter as well as the frequency and phase estimation errors at the nodes are included and modeled using practical statistics. A decentralized frequency and phase consensus (DFPC) algorithm is proposed which uses an average consensus method in which each node in the array iteratively updates its frequency and phase by computing an average of the frequencies and phases of their neighboring nodes. Simulation results show that upon convergence the DFPC algorithm can align the frequencies and phases of all the nodes up to a residual phase error that is governed by the oscillators and the estimation errors. To reduce this residual phase error and thus improve the synchronization between the nodes, a Kalman filter based decentralized frequency and phase consensus (KF-DFPC) algorithm is presented. The total residual phase error at the convergence of the KF-DFPC and DFPC algorithms is derived theoretically. The synchronization performances of these algorithms are compared to each other and to the diffusion least mean square (DLMS), the diffusion KF (DKF) algorithm, and the Kalman consensus information filter (KCIF) algorithms, in light of this theoretical residual phase error by varying the duration of the signals, connectivity of the nodes, the number of nodes in the array, and signal to noise ratio of the received signals. Simulation results demonstrate that under certain conditions the proposed KF-DFPC algorithm outperforms and converges in fewer iterations than all the algorithms. Furthermore, for shorter intervals between local information broadcasts, the KF-DFPC algorithm significantly outperforms the DFPC algorithm in reducing the residual total phase error, irrespective of the signal to noise ratio of the received signals.