Miracle-dz/UniFAR-FTUs huggingface dataset:
AOSLO-net: A Deep Learning-Based Method for Automatic Segmentation of Retinal Microaneurysms From Adaptive Optics Scanning Laser Ophthalmoscopy Images

Purpose Accurate segmentation of microaneurysms (MAs) from adaptive optics scanning laser ophthalmoscopy (AOSLO) images is crucial for identifying MA morphologies and assessing the hemodynamics inside the MAs. Herein, we introduce AOSLO-net to perform automatic MA segmentation from AOSLO images of diabetic retinas. Method AOSLO-net is composed of a deep neural network based on UNet with a pretrained EfficientNet as the encoder. We have designed customized preprocessing and postprocessing policies for AOSLO images, including generation of multichannel images, de-noising, contrast enhancement, ensemble and union of model predictions, to optimize the MA segmentation. AOSLO-net is trained and tested using 87 MAs imaged from 28 eyes of 20 subjects with varying severity of diabetic retinopathy (DR), which is the largest available AOSLO dataset for MA detection. To avoid the overfitting in the model training process, we augment the training data by flipping, rotating, scaling the original image to increase the diversity of data available for model training. Results The validity of the model is demonstrated by the good agreement between the predictions of AOSLO-net and the MA masks generated by ophthalmologists and skillful trainees on 87 patient-specific MA images. Our results show that AOSLO-net outperforms the state-of-the-art segmentation model (nnUNet) both in accuracy (e.g., intersection over union and Dice scores), as well as computational cost. Conclusions We demonstrate that AOSLO-net provides high-quality of MA segmentation from AOSLO images that enables correct MA morphological classification. Translational Relevance As the first attempt to automatically segment retinal MAs from AOSLO images, AOSLO-net could facilitate the pathological study of DR and help ophthalmologists make disease prognoses.
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization

Many popular learning-rate schedules for deep neural networks combine a decaying trend with local perturbations that attempt to escape saddle points and bad local minima. We derive convergence guarantees for bandwidth-based step-sizes, a general class of learning-rates that are allowed to vary in a banded region. This framework includes cyclic and non-monotonic step-sizes for which no theoretical guarantees were previously known. We provide worst-case guarantees for SGD on smooth non-convex problems under several bandwidth-based step sizes, including stagewise $1/\sqrt{t}$ and the popular step-decay (constant and then drop by a constant), which is also shown to be optimal. Moreover, we show that its momentum variant (SGDM) converges as fast as SGD with the bandwidth-based step-decay step-size. Finally, we propose some novel step-size schemes in the bandwidth-based family and verify their efficiency on several deep neural network training tasks.
Ensemble Defense with Data Diversity: Weak Correlation Implies Strong Robustness

In this paper, we propose a framework of filter-based ensemble of deep neuralnetworks (DNNs) to defend against adversarial attacks. The framework builds an ensemble of sub-models -- DNNs with differentiated preprocessing filters. From the theoretical perspective of DNN robustness, we argue that under the assumption of high quality of the filters, the weaker the correlations of the sensitivity of the filters are, the more robust the ensemble model tends to be, and this is corroborated by the experiments of transfer-based attacks. Correspondingly, we propose a principle that chooses the specific filters with smaller Pearson correlation coefficients, which ensures the diversity of the inputs received by DNNs, as well as the effectiveness of the entire framework against attacks. Our ensemble models are more robust than those constructed by previous defense methods like adversarial training, and even competitive with the classical ensemble of adversarial trained DNNs under adversarial attacks when the attacking radius is large.
Unsupervised Action Segmentation for Instructional Videos

In this paper we address the problem of automatically discovering atomic actions in unsupervised manner from instructional videos, which are rarely annotated with atomic actions. We present an unsupervised approach to learn atomic actions of structured human tasks from a variety of instructional videos based on a sequential stochastic autoregressive model for temporal segmentation of videos. This learns to represent and discover the sequential relationship between different atomic actions of the task, and which provides automatic and unsupervised self-labeling.
Non-delay limit in the energy space from the nonlinear damped wave equation to the nonlinear heat equation

We consider a singular limit problem from the damped wave equation with a power type nonlinearity (NLDW) to the corresponding heat equation (NLH). We call our singular limit problem non-delay limit. We show that the solution of NLDW goes to the one of NLH in [Formula: see text] topology under the both [Formula: see text] regularity solutions. We also obtain the positive convergence rate in the weaker topology [Formula: see text]. Moreover, with restriction of the range of power, if the solution to NLH is global and decays to zero, then we get the global-in-time uniform convergence of the non-delay limit.
DL-DDA - Deep Learning based Dynamic Difficulty Adjustment with UX and Gameplay constraints

Dynamic difficulty adjustment (DDA) is a process of automatically changing a game difficulty for optimization of the user experience. It is a vital part of almost any modern game. Most existing DDA approaches concentrate on the experience of a player without looking at the rest of the players. We propose a method that automatically optimizes the user experience while taking into consideration other players and the macro constraints imposed by the game. The method is based on a deep neural network architecture that involves a count loss constraint that has zero gradients in most of its support. We suggest a method to optimize this loss function and provide theoretical analysis of its performance. Finally, we provide empirical results of an internal experiment that was done on 200,000 players and was found to outperform the corresponding manual heuristics crafted by game design experts.
Galaxy Morphologies Revealed with Subaru HSC and Super-Resolution Techniques I: Major Merger Fractions of L_UV~3-15 L_UV* Dropout Galaxies at z~4-7

We perform a super-resolution analysis of the Subaru Hyper Suprime-Cam (HSC) images to estimate the major merger fractions of z~4-7 dropout galaxies at the bright end of galaxy UV luminosity functions (LFs). Our super-resolution technique improves the spatial resolution of the ground-based HSC images, from ~1" to ~90%. We apply the super-resolution technique to 6535 very bright dropout galaxies in a UV luminosity range of L_UV~3-15 L_UV* corresponding to -24 ~10 L_UV*, even after the AGN contribution is subtracted. The results support scenarios in which other additional mechanisms such as insignificant mass quenching effects and the low dust obscuration contribute to the DPL shape of galaxy UV LFs.
Scientific Dataset Discovery via Topic-level Recommendation

Data intensive research requires the support of appropriate datasets. However, it is often time-consuming to discover usable datasets matching a specific research topic. We formulate the dataset discovery problem on an attributed heterogeneous graph, which is composed of paper-paper citation, paper-dataset citation, and also paper content. We propose to characterize both paper and dataset nodes by their commonly shared latent topics, rather than learning user and item representations via canonical graph embedding models, because the usage of datasets and the themes of research projects can be understood on the common base of research topics. The relevant datasets to a given research project can then be inferred in the shared topic space. The experimental results show that our model can generate reasonable profiles for datasets, and recommend proper datasets for a query, which represents a research project linked with several papers.
Churn Reduction via Distillation

In real-world systems, models are frequently updated as more data becomes available, and in addition to achieving high accuracy, the goal is to also maintain a low difference in predictions compared to the base model (i.e. predictive “churn”). If model retraining results in vastly different behavior, then it could cause negative effects in downstream systems, especially if this churn can be avoided with limited impact on model accuracy. In this paper, we show an equivalence between training with distillation using the base model as the teacher and training with an explicit constraint on the predictive churn. We then show that distillation performs strongly for low churn training against a number of recent baselines on a wide range of datasets and model architectures, including fully-connected networks, convolutional networks, and transformers.
Self-Supervised Graph Learning with Proximity-based Views and Channel Contrast

We consider graph representation learning in a self-supervised manner. Graph neural networks (GNNs) use neighborhood aggregation as a core component that results in feature smoothing among nodes in proximity. While successful in various prediction tasks, such a paradigm falls short of capturing nodes' similarities over a long distance, which proves to be important for high-quality learning. To tackle this problem, we strengthen the graph with two additional graph views, in which nodes are directly linked to those with the most similar features or local structures. Not restricted by connectivity in the original graph, the generated views allow the model to enhance its expressive power with new and complementary perspectives from which to look at the relationship between nodes. Following a contrastive learning approach, we propose a method that aims to maximize the agreement between representations across generated views and the original graph. We also propose a channel-level contrast approach that greatly reduces computation cost, compared to the commonly used node level contrast, which requires computation cost quadratic in the number of nodes. Extensive experiments on seven assortative graphs and four disassortative graphs demonstrate the effectiveness of our approach.
Signatured Deep Fictitious Play for Mean Field Games with Common Noise

Existing deep learning methods for solving mean-field games (MFGs) with common noise fix the sampling common noise paths and then solve the corresponding MFGs. This leads to a nested-loop structure with millions of simulations of common noise paths in order to produce accurate solutions, which results in prohibitive computational cost and limits the applications to a large extent. In this paper, based on the rough path theory, we propose a novel single-loop algorithm, named signatured deep fictitious play, by which we can work with the unfixed common noise setup to avoid the nested-loop structure and reduce the computational complexity significantly. The proposed algorithm can accurately capture the effect of common uncertainty changes on mean-field equilibria without further training of neural networks, as previously needed in the existing machine learning algorithms. The efficiency is supported by three applications, including linear-quadratic MFGs, mean-field portfolio game, and mean-field game of optimal consumption and investment. Overall, we provide a new point of view from the rough path theory to solve MFGs with common noise with significantly improved efficiency and an extensive range of applications. In addition, we report the first deep learning work to deal with extended MFGs (a mean-field interaction via both the states and controls) with common noise.
Representation mitosis in wide neural networks

Deep neural networks (DNNs) defy the classical bias-variance trade-off: adding parameters to a DNN that exactly interpolates its training data will typically improve its generalisation performance. Explaining the mechanism behind the beneﬁt of such over-parameterisation is an outstanding challenge for deep learning theory. Here, we study the last layer representation of various deep architectures such as Wide-ResNets for image classiﬁcation and ﬁnd evidence for an underlying mechanism that we call representation mitosis : if the last hidden representation is wide enough, its neurons tend to split into groups which carry identical information, and differ from each other only by a statistically independent noise. Like in a mitosis process, the number of such groups, or “clones”, increases linearly with the width of the layer, but only if the width is above a critical value. We show that a key ingredient to activate mitosis is continuing the training process until the training error is zero. Finally, we show that in one of the learning tasks we considered, a wide model with several automatically developed clones performs signiﬁcantly better than a deep ensemble based on architectures in which the last layer has the same size as the clones.
Recovery Analysis for Plug-and-Play Priors using the Restricted Eigenvalue Condition

The plug-and-play priors (PnP) and regularization by denoising (RED) methods have become widely used for solving inverse problems by leveraging pre-trained deep denoisers as image priors. While the empirical imaging performance and the theoretical convergence properties of these algorithms have been widely investigated, their recovery properties have not previously been theoretically analyzed. We address this gap by showing how to establish theoretical recovery guarantees for PnP/RED by assuming that the solution of these methods lies near the fixed-points of a deep neural network. We also present numerical results comparing the recovery performance of PnP/RED in compressive sensing against that of recent compressive sensing algorithms based on generative models. Our numerical results suggest that PnP with a pre-trained artifact removal network provides significantly better results compared to the existing state-of-the-art methods.
Long-term and multiwavelength evolution of a changing-look AGN Mrk 1018

The physical mechanism for triggering the changing-look phenomenon in active galactic nuclei (AGNs) is still unclear. We explore this issue based on the multi-wavelength spectral and flux variations for a changing-look AGN Mrk 1018 with long-term observations in the X-ray, optical/ultraviolet(UV), and radio bands. Both the optical and the X-ray emission experience rapid decay in changing-look phase during 2010–2015, where a re-flare appears in the optical/UV and X-ray bands. We find a time lag of ∼ 20 days of optical/UV behind X-ray variations in type 1.9 phase. The 5 GHz radio flux decreases by∼ 20% in type 1.9 phase during 2016–2017. We find both X-ray photon index (Γ) and the optical-to-X-ray spectral index (αox ) are anti-correlated with the Eddington scaled 2–10 keVX-ray luminosity (LX/LEdd) in the type 1.9 phase. However, the type 1 phase deviates from these two anti-correlations, which suggests that the change of broad emission lines might be regulated by the evolution of accretion disk (e.g., disappearing of the inner cold disk in the type 1.9 phase).
Hyperspectral imaging with Raman scattered photons: A new paradigm in Raman analysis

Surface enhanced Raman spectroscopy, is a technique of fundamental importance to analytical science and technology where the amplified Raman spectrum of analytes is used for chemical fingerprinting. Here, we showcase an engineered hierarchical substrate in which the plasmonically active regions are restricted to a micron scale, 2D hexagonal pattern. The Raman signal enhancement of any analyte uniformly coating the substrate is consequently bears a high registry with the 2D pattern. This spatially segregated enhancement allows optical imaging of the 2D pattern solely using the Raman scattered photons from the analyte. While pattern brightness and contrast determine analyte identification and detection sensitivity, the spectrally selective contrast allows for tuning specificity. Conceptual proof of the technique is demonstrated via the acquisition of Raman images with rhodamine and fluorescein and finally applied to detect glucose in 40 mM concentration. The large area imaging and the inherent requirement of spatial uniformity for positive detection implemented using a machine learning based automated pattern recognition protocol increases the statistical confidence of analyte detection. This simultaneously multisite signal detection sacrifices continuous spectral information at the cost of speed, reproducibility and human error via automation of detection of the hyperspectral imaging technique presented here.
Denoising Word Embeddings by Averaging in a Shared Space

We introduce a new approach for smoothing and improving the quality of word embeddings. We consider a method of fusing word embeddings that were trained on the same corpus but with different initializations. We project all the models to a shared vector space using an efficient implementation of the Generalized Procrustes Analysis (GPA) procedure, previously used in multilingual word translation. Our word representation demonstrates consistent improvements over the raw models as well as their simplistic average, on a range of tasks. As the new representations are more stable and reliable, there is a noticeable improvement in rare word evaluations.
Constrained Generalized Additive 2 Model With Consideration of High-Order Interactions

In recent years, machine learning and AI have been introduced in many industrial fields. In fields such as finance, medicine, and autonomous driving, where the inference results of a model may have serious consequences, high interpretability, as well as prediction accuracy is required. In this study, we propose Constrained Generalized Additive 2 Model plus (CGA2M+), which is based on the Generalized Additive 2 Model (GA2M) and differs from it in two major points. The first is the introduction of monotonicity. Enforcing monotonicity constraints on some features using an analyst's knowledge, is expected to improve not only interpretability but also generalization performance. The second is the introduction of a higher-order term: given that GA2M considers only up to second-order interactions, we introduce a higher-order term, which can capture higher-order interactions and improve prediction performance. In this process, the higher-order term does not reduce interpretability by applying our training method. Numerical experiments showed that the proposed model has high prediction accuracy and interpretability. Furthermore, we confirmed that generalization performance is improved by imposing monotonicity constraints.
Accelerating Stochastic Simulation with Interactive Neural Processes

Stochastic simulations such as large-scale, spatiotemporal, age-structured epidemic models are computationally expensive at ﬁne-grained resolution. We propose Interactive Neural Process ( INP ), an interactive framework to continuously learn a deep learning surrogate model and accelerate simulation. Our framework is based on the novel integration of Bayesian active learning, stochastic simulation and deep sequence modeling. In particular, we develop a novel spatiotemporal neural process model to mimic the underlying process dynamics. Our model automatically infers the latent process which describes the intrinsic uncertainty of the simulator. This also gives rise to a new acquisition function that can quantify the uncertainty of deep learning predictions. We design Bayesian active learning algorithms to iteratively query the simulator, gather more data, and continuously improve the model. We perform theoretical analysis and demonstrate that our approach reduces sample complexity compared with random sampling in high dimension. Empirically, we demonstrate our framework can faithfully imitate the behavior of a complex infectious disease simulator with a small number of examples, enabling rapid simulation and scenario exploration.
Robust Stochastic Linear Contextual Bandits Under Adversarial Attacks

Stochastic linear contextual bandit algorithms have substantial applications in practice, such as recommender systems, online advertising, clinical trials, etc. Recent works show that optimal bandit algorithms are vulnerable to adversarial attacks and can fail completely in the presence of attacks. Existing robust bandit algorithms only work for the non-contextual setting under the attack of rewards and cannot improve the robustness in the general and popular contextual bandit environment. In addition, none of the existing methods can defend against attacked context. In this work, we provide the first robust bandit algorithm for stochastic linear contextual bandit setting under a fully adaptive and omniscient attack with sub-linear regret. Our algorithm not only works under the attack of rewards, but also under attacked context. Moreover, it does not need any information about the attack budget or the particular form of the attack. We provide theoretical guarantees for our proposed algorithm and show by experiments that our proposed algorithm improves the robustness against various kinds of popular attacks.
Possibility of multi-step electroweak phase transition in the two Higgs doublet models

We discuss whether a multi-step electroweak phase transition (EWPT) occurs in two Higgs doublet models (2HDMs). The EWPT is related to interesting phenomena such as baryogenesis and a gravitational wave from it. We examine parameter regions in CP-conserving 2HDMs and ﬁnd certain areas where the multi-step EWPTs occur. The parameter search shows the multi-step EWPT prefers the scalar potential with the approximate Z 2 symmetry and a mass hierarchy between the neutral CP-odd and CP-even extra scalar bosons m A < m H . By contrast, the multi-step EWPT whose ﬁrst step is strongly ﬁrst order fa-vors a mass hierarchy m A > m H . In addition, we compute the Higgs trilinear coupling in the parameter region where the multi-step EWPTs occur, which can be observed at future colliders. We also discuss a multi-peaked gravitational wave from a multi-step EWPT.
Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularization

There have been many recent advances in representation learning; however, unsupervised representation learning can still struggle with model identification issues related to rotations of the latent space. Variational Auto-Encoders (VAEs) and their extensions such as $\beta$-VAEs have been shown to improve local alignment of latent variables with PCA directions, which can help to improve model disentanglement under some conditions. Borrowing inspiration from Independent Component Analysis (ICA) and sparse coding, we propose applying an $L_1$ loss to the VAE's generative Jacobian during training to encourage local latent variable alignment with independent factors of variation in images of multiple objects or images with multiple parts. We demonstrate our results on a variety of datasets, giving qualitative and quantitative results using information theoretic and modularity measures that show our added $L_1$ cost encourages local axis alignment of the latent representation with individual factors of variation.
Technical Report: Temporal Aggregate Representations

This technical report extends our work presented in [9] with more experiments. In [9], we tackle long-term video understanding, which requires reasoning from current and past or future observations and raises several fundamental questions. How should temporal or sequential relationships be modelled? What temporal extent of information and context needs to be processed? At what temporal scale should they be derived? [9] addresses these questions with a flexible multi-granular temporal aggregation framework. In this report, we conduct further experiments with this framework on different tasks and a new dataset, EPIC-KITCHENS-100. Our code and models can be found in https://github.com/dibschat/tempAgg