Miracle-dz/UniFAR-FTUs huggingface dataset:
The nature of faint radio galaxies at high redshifts

We present spectra and near-infrared images of a sample of faint radio sources initially selected as promising high-redshift radio galaxy (HzRG) candidates. We have determined redshifts for a total of 13 radio galaxies with redshifts ranging from 0.52 ≤ $z$ ≤ 5.72. Our sample probes radio luminosities that are almost an order of magnitude fainter than previous large samples at the highest redshifts. We use near-infrared photometry for a subsample of these galaxies to calculate stellar masses using simple stellar population models, and find stellar masses to be in the range $10^{10.8} {--}10^{11.7} \, \mathrm{M}_\odot$. We then compare our faint radio galaxies with brighter radio galaxies at $z$ ≥ 2 from the literature. We find that fainter radio galaxies have lower Ly α luminosities and narrower line widths compared to the bright ones, implying photoionization by a weaker active galactic nucleus (AGN). We also rule out the presence of strong shocks in faint HzRGs. The stellar masses determined for faint HzRGs are lower than those observed for brighter ones. We find that faint HzRG population in the redshift range 2–4 forms a bridge between star-forming and narrow-line AGNs, whereas the ones at $z$ > 4 are likely to be dominated by star formation, and may be building up their stellar mass through cold accretion of gas. Finally, we show that the overall redshift evolution of radio sizes at $z$ > 2 is fully compatible with increased inverse Compton scattering losses at high redshifts.
Disparate Vulnerability: on the Unfairness of Privacy Attacks Against Machine Learning

A membership inference attack (MIA) against a machine learning model enables an attacker to determine whether a given data record was part of the model's training data or not. The effectiveness of these attacks is reported using metrics computed across the whole population (e.g., average attack accuracy). In this paper, we show that the attack success varies across different subgroups of the data (e.g., race, gender), i.e., there is \emph{disparate vulnerability}. Even if MIA's success looks no better than random guessing over the whole population, subgroups can still be vulnerable. We study the necessary and sufficient conditions for a classifier to exhibit disparate vulnerability, and we determine to what extent certain learning techniques (e.g., fairness constraints, differential privacy) can prevent it. Our work provides a theoretical framework for studying MIA attacks from a new perspective.
A subjective method for the measurement of monochromatic aberrations of the eye.

We have designed an aberroscope that differs from Tscherning’s classical instrument in that it makes use of an artificial astigmatism rather than an artificial myopia to defocus the image of a point source of light. A subject views the source through a ±5 D crossed cylinder lens with axes at 45° to the principal axes of an intercalated grid and sees a shadow image of the grid. The distortions of this grid image are quantitatively related to the wave aberration of the eye. Using this device we have obtained drawings for more than 50 subjects. These drawings of the grid pattern have been analyzed by means of a two-dimensional polynomial curve Fitting technique that computes Taylor polynomial terms to the fourth order. From the Taylor coefficients it is possible to reconstruct the wave aberration surface. Examination of the Taylor terms so obtained shows that the monochromatic aberrations of the eye are dominated by third-order Taylor terms within the range of physiological pupil sizes, and that spherical aberration frequently appears predominantly about one axis only, a condition that we have termed “cylindrical” aberration. We have computed the optical MTF of our subjects’ eyes and find that the role of aberrations in degrading the MTF may be greater than generally believed.
Application Components Migration in NFV-based Hybrid Cloud/Fog Systems

Fog computing extends the cloud to the edge of the network, close to the end-users enabling the deployment of some application component in the fog while others in the cloud. Network Functions Virtualization (NFV) decouples the network functions from the underlying hardware. In NFV settings, application components can be implemented as sets of Virtual Network Functions (VNFs) chained in specific order representing VNF-Forwarding Graphs (VNF-FG). Many studies have been carried out to map the VNF-FGs to cloud systems. However, in hybrid cloud/fog systems, an additional challenge arises. The mobility of fog nodes may cause high latency as the distance between the end-users and the nodes hosting the components increases. This may not be tolerable for some applications. In such cases, a prominent solution is to migrate application components to a closer fog node. This paper focuses on application component migration in NFV-based hybrid cloud/fog systems. The objective is to minimize the aggregated makespan of the applications. The problem is modeled mathematically, and a heuristic is proposed to find the sub-optimal solution in an acceptable time. The heuristic aims at finding the optimal fog node in each time-slot considering a pre-knowledge of the mobility models of the fog nodes. The experiment’s results show that our proposed solution improves the makespan and the number of migrations compared to random migration and No-migration.
Pangloss: a novel Markov chain prefetcher

We present Pangloss, an efficient high-performance data prefetcher that approximates a Markov chain on delta transitions. With a limited information scope and space/logic complexity, it is able to reconstruct a variety of both simple and complex access patterns. This is achieved by a highly-efficient representation of the Markov chain to provide accurate values for transition probabilities. In addition, we have added a mechanism to reconstruct delta transitions originally obfuscated by the out-of-order execution or page transitions, such as when streaming data from multiple sources. Our single-level (L2) prefetcher achieves a geometric speedup of 1.7% and 3.2% over selected state-of-the-art baselines (KPCP and BOP). When combined with an equivalent for the L1 cache (L1 & L2), the speedups rise to 6.8% and 8.4%, and 40.4% over non-prefetch. In the multi-core evaluation, there seems to be a considerable performance improvement as well.
Patient-Specific Effects of Medication Using Latent Force Models with Gaussian Processes

Multi-output Gaussian processes (GPs) are a flexible Bayesian nonparametric framework that has proven useful in jointly modeling the physiological states of patients in medical time series data. However, capturing the short-term effects of drugs and therapeutic interventions on patient physiological state remains challenging. We propose a novel approach that models the effect of interventions as a hybrid Gaussian process composed of a GP capturing patient physiology convolved with a latent force model capturing effects of treatments on specific physiological features. This convolution of a multi-output GP with a GP including a causal time-marked kernel leads to a well-characterized model of the patients' physiological state responding to interventions. We show that our model leads to analytically tractable cross-covariance functions, allowing scalable inference. Our hierarchical model includes estimates of patient-specific effects but allows sharing of support across patients. Our approach achieves competitive predictive performance on challenging hospital data, where we recover patient-specific response to the administration of three common drugs: one antihypertensive drug and two anticoagulants.
Quantitatively ranking incorrect responses to multiple-choice questions using item response theory

Research-based assessment instruments (RBAIs) are ubiquitous throughout both physics instruction and physics education research. The vast majority of analyses involving student responses to RBAI questions have focused on whether or not a student selects correct answers and using correctness to measure growth. This approach often undervalues the rich information that may be obtained by examining students' particular choices of incorrect answers. In the present study, we aim to reveal some of this valuable information by quantitatively determining the relative correctness of various incorrect responses. To accomplish this, we propose an assumption that allow us to define relative correctness: students who have a high understanding of Newtonian physics are likely to answer more questions correctly and also more likely to choose better incorrect responses, than students who have a low understanding. Analyses using item response theory align with this assumption, and Bock's nominal response model allows us to uniquely rank each incorrect response. We present results from over 7,000 students' responses to the Force and Motion Conceptual Evaluation.
A mean-field limit for certain deep neural networks

Understanding deep neural networks (DNNs) is a key challenge in the theory of machine learning, with potential applications to the many fields where DNNs have been successfully used. This article presents a scaling limit for a DNN being trained by stochastic gradient descent. Our networks have a fixed (but arbitrary) number $L\geq 2$ of inner layers; $N\gg 1$ neurons per layer; full connections between layers; and fixed weights (or "random features" that are not trained) near the input and output. Our results describe the evolution of the DNN during training in the limit when $N\to +\infty$, which we relate to a mean field model of McKean-Vlasov type. Specifically, we show that network weights are approximated by certain "ideal particles" whose distribution and dependencies are described by the mean-field model. A key part of the proof is to show existence and uniqueness for our McKean-Vlasov problem, which does not seem to be amenable to existing theory. Our paper extends previous work on the $L=1$ case by Mei, Montanari and Nguyen; Rotskoff and Vanden-Eijnden; and Sirignano and Spiliopoulos. We also complement recent independent work on $L>1$ by Sirignano and Spiliopoulos (who consider a less natural scaling limit) and Nguyen (who nonrigorously derives similar results).
Unsupervised Neural Generative Semantic Hashing

Fast similarity search is a key component in large-scale information retrieval, where semantic hashing has become a popular strategy for representing documents as binary hash codes. Recent advances in this area have been obtained through neural network based models: generative models trained by learning to reconstruct the original documents. We present a novel unsupervised generative semantic hashing approach, Ranking based Semantic Hashing (RBSH) that consists of both a variational and a ranking based component. Similarly to variational autoencoders, the variational component is trained to reconstruct the original document conditioned on its generated hash code, and as in prior work, it only considers documents individually. The ranking component solves this limitation by incorporating inter-document similarity into the hash code generation, modelling document ranking through a hinge loss. To circumvent the need for labelled data to compute the hinge loss, we use a weak labeller and thus keep the approach fully unsupervised. Extensive experimental evaluation on four publicly available datasets against traditional baselines and recent state-of-the-art methods for semantic hashing shows that RBSH significantly outperforms all other methods across all evaluated hash code lengths. In fact, RBSH hash codes are able to perform similarly to state-of-the-art hash codes while using 2-4x fewer bits.
Variability of young stellar objects in the star-forming region Pelican Nebula

Context. Pre-main-sequence variability characteristics can be used to probe the physical processes leading to the formation and initial evolution of both stars and planets. Aims. The photometric variability of pre-main-sequence stars is studied at optical wavelengths to explore star–disk interactions, accretion, spots, and other physical mechanisms associated with young stellar objects. Methods. We observed a field of 16′ × 16′ in the star-forming region Pelican Nebula (IC 5070) at BVRI wavelengths for 90 nights spread over one year in 2012−2013. More than 250 epochs in the VRI bands are used to identify and classify variables up to V ∼ 21 mag. Their physical association with the cluster IC 5070 is established based on the parallaxes and proper motions from the Gaia second data release (DR2). Multiwavelength photometric data are used to estimate physical parameters based on the isochrone fitting and spectral energy distributions. Results. We present a catalog of optical time-series photometry with periods, mean magnitudes, and classifications for 95 variable stars including 67 pre-main-sequence variables towards star-forming region IC 5070. The pre-main-sequence variables are further classified as candidate classical T Tauri and weak-line T Tauri stars based on their light curve variations and the locations on the color-color and color-magnitude diagrams using optical and infrared data together with Gaia DR2 astrometry. Classical T Tauri stars display variability amplitudes up to three times the maximum fluctuation in disk-free weak-line T Tauri stars, which show strong periodic variations. Short-term variability is missed in our photometry within single nights. Several classical T Tauri stars display long-lasting (≥10 days) single or multiple fading and brightening events of up to two magnitudes at optical wavelengths. The typical mass and age of the pre-main-sequence variables from the isochrone fitting and spectral energy distributions are estimated to be ≤1 M⊙ and ∼2 Myr, respectively. We do not find any correlation between the optical amplitudes or periods with the physical parameters (mass and age) of pre-main-sequence stars. Conclusions. The low-mass pre-main-sequence stars in the Pelican Nebula region display distinct variability and color trends and nearly 30% of the variables exhibit strong periodic signatures attributed to cold spot modulations. In the case of accretion bursts and extinction events, the average amplitudes are larger than one magnitude at optical wavelengths. These optical magnitude fluctuations are stable on a timescale of one year.
Accurate Modeling of the Projected Galaxy Clustering in Photometric Surveys. I. Tests with Mock Catalogs

We develop a novel method to explore the galaxy–halo connection using the galaxy imaging surveys by modeling the projected two-point correlation function measured from the galaxies with reasonable photometric redshift measurements. By assuming a Gaussian form of the photometric redshift errors, we are able to simultaneously constrain the halo occupation distribution (HOD) models and the effective photometric redshift uncertainties. Tests with mock galaxy catalogs demonstrate that this method can successfully recover (within ∼1σ) the intrinsic large-scale galaxy bias, as well as the HOD models and the effective photometric redshift uncertainty. This method also works well even for galaxy samples with 10% catastrophic photometric redshift errors.
Deep Face Recognition Model Compression via Knowledge Transfer and Distillation

Fully convolutional networks (FCNs) have become de facto tool to achieve very high-level performance for many vision and non-vision tasks in general and face recognition in particular. Such high-level accuracies are normally obtained by very deep networks or their ensemble. However, deploying such high performing models to resource constraint devices or real-time applications is challenging. In this paper, we present a novel model compression approach based on student-teacher paradigm for face recognition applications. The proposed approach consists of training teacher FCN at bigger image resolution while student FCNs are trained at lower image resolutions than that of teacher FCN. We explored three different approaches to train student FCNs: knowledge transfer (KT), knowledge distillation (KD) and their combination. Experimental evaluation on LFW and IJB-C datasets demonstrate comparable improvements in accuracies with these approaches. Training low-resolution student FCNs from higher resolution teacher offer fourfold advantage of accelerated training, accelerated inference, reduced memory requirements and improved accuracies. We evaluated all models on IJB-C dataset and achieved state-of-the-art results on this benchmark. The teacher network and some student networks even achieved Top-1 performance on IJB-C dataset. The proposed approach is simple and hardware friendly, thus enables the deployment of high performing face recognition deep models to resource constraint devices.
ArcticNet: A Deep Learning Solution to Classify Arctic Wetlands

Arctic environments are rapidly changing under the warming climate. Of particular interest are wetlands, a type of ecosystem that constitutes the most effective terrestrial long-term carbon store. As permafrost thaws, the carbon that was locked in these wetland soils for millennia becomes available for aerobic and anaerobic decomposition, which releases CO2 and CH4, respectively, back to the atmosphere.As CO2 and CH4 are potent greenhouse gases, this transfer of carbon from the land to the atmosphere further contributes to global warming, thereby increasing the rate of permafrost degradation in a positive feedback loop. Therefore, monitoring Arctic wetland health and dynamics is a key scientific task that is also of importance for policy. However, the identification and delineation of these important wetland ecosystems, remain incomplete and often inaccurate. Mapping the extent of Arctic wetlands remains a challenge for the scientific community. Conventional, coarser remote sensing methods are inadequate at distinguishing the diverse and micro-topographically complex non-vascular vegetation that characterize Arctic wetlands, presenting the need for better identification methods. To tackle this challenging problem, we constructed and annotated the first-of-its-kind Arctic Wetland Dataset (AWD). Based on that, we present ArcticNet, a deep neural network that exploits the multi-spectral, high-resolution imagery captured from nanosatellites (Planet Dove CubeSats) with additional DEM from the ArcticDEM project, to semantically label a Arctic study area into six types, in which three Arctic wetland functional types are included. We present multi-fold efforts to handle the arising challenges, including class imbalance, and the choice of fusion strategies. Preliminary results endorse the high promise of ArcticNet, achieving 93.12% in labelling a hold-out set of regions in our Arctic study area.
Sample-Optimal Low-Rank Approximation of Distance Matrices

A distance matrix $A \in \mathbb R^{n \times m}$ represents all pairwise distances, $A_{ij}=\mathrm{d}(x_i,y_j)$, between two point sets $x_1,. ,x_n$ and $y_1,. ,y_m$ in an arbitrary metric space $(\mathcal Z, \mathrm{d})$. Such matrices arise in various computational contexts such as learning image manifolds, handwriting recognition, and multi-dimensional unfolding. In this work we study algorithms for low-rank approximation of distance matrices. Recent work by Bakshi and Woodruff (NeurIPS 2018) showed it is possible to compute a rank-$k$ approximation of a distance matrix in time $O((n+m)^{1+\gamma}) \cdot \mathrm{poly}(k,1/\epsilon)$, where $\epsilon>0$ is an error parameter and $\gamma>0$ is an arbitrarily small constant. Notably, their bound is sublinear in the matrix size, which is unachievable for general matrices. We present an algorithm that is both simpler and more efficient. It reads only $O((n+m) k/\epsilon)$ entries of the input matrix, and has a running time of $O(n+m) \cdot \mathrm{poly}(k,1/\epsilon)$. We complement the sample complexity of our algorithm with a matching lower bound on the number of entries that must be read by any algorithm. We provide experimental results to validate the approximation quality and running time of our algorithm.
Sparsity Normalization: Stabilizing the Expected Outputs of Deep Networks

The learning of deep models, in which a numerous of parameters are superimposed, is known to be a fairly sensitive process and should be carefully done through a combination of several techniques that can help to stabilize it. We introduce an additional challenge that has never been explicitly studied: the heterogeneity of sparsity at the instance level due to missing values or the innate nature of the input distribution. We confirm experimentally on the widely used benchmark datasets that this variable sparsity problem makes the output statistics of neurons unstable and makes the learning process more difficult by saturating non-linearities. We also provide the analysis of this phenomenon, and based on our analysis, we present a simple technique to prevent this issue, referred to as Sparsity Normalization (SN). Finally, we show that the performance can be significantly improved with SN on certain popular benchmark datasets, or that similar performance can be achieved with lower capacity. Especially focusing on the collaborative filtering problem where the variable sparsity issue has been completely ignored, we achieve new state-of-the-art results on Movielens 100k and 1M datasets, by simply applying Sparsity Normalization (SN).
Monotone relations in Hadamard spaces

In this paper, the notion of W-property for subsets of X x X? is introduced and investigated, where X is an Hadamard space and X? is its linear dual space. It is shown that an Hadamard space X is flat if and only if X x X? has W-property. Moreover, the notion of monotone relation from an Hadamard space to its linear dual space is introduced. A characterization result for monotone relations with W-property (and hence in flat Hadamard spaces) is given. Finally, a type of Debrunner-Flor Lemma concerning extension of monotone relations in Hadamard spaces is proved.
DAWN: Dynamic Adversarial Watermarking of Neural Networks

Training machine learning (ML) models is expensive in terms of computational power, amounts of labeled data and human expertise. Thus, ML models constitute business value for their owners. Embedding digital watermarks during model training allows a model owner to later identify their models in case of theft or misuse. However, model functionality can also be stolen via model extraction, where an adversary trains a surrogate model using results returned from a prediction API of the original model. Recent work has shown that model extraction is a realistic threat. Existing watermarking schemes are ineffective against model extraction since it is the adversary who trains the surrogate model. In this paper, we introduce DAWN (Dynamic Adversarial Watermarking of Neural Networks), the first approach to use watermarking to deter model extraction theft. Unlike prior watermarking schemes, DAWN does not impose changes to the training process but operates at the prediction API of the protected model, by dynamically changing the responses for a small subset of queries (e.g., 0.5%) from API clients. This set is a watermark that will be embedded in case a client uses its queries to train a surrogate model. We show that DAWN is resilient against two state-of-the-art model extraction attacks, effectively watermarking all extracted surrogate models, allowing model owners to reliably demonstrate ownership (with confidence greater than 1-2-64), incurring negligible loss of prediction accuracy (0.03-0.5%).