Miracle-dz/UniFAR-FTUs huggingface dataset:
Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants

Recent progress in autoencoder-based sparse identification of nonlinear dynamics (SINDy) under ℓ 1 constraints allows joint discoveries of governing equations and latent coordinate systems from spatio-temporal data, including simulated video frames. However, it is challenging for ℓ 1 -based sparse inference to perform correct identification for real data due to the noisy measurements and often limited sample sizes. To address the data-driven discovery of physics in the low-data and high-noise regimes, we propose Bayesian SINDy autoencoders, which incorporate a hierarchical Bayesian Spike-and-slab Gaussian Lasso prior. Bayesian SINDy autoencoder enables the joint discovery of governing equations and coordinate systems with uncertainty estimate. To resolve the challenging computational tractability of the Bayesian hierarchical setting, we adapt an adaptive empirical Bayesian method with Stochastic Gradient Langevin Dynamics (SGLD) which gives a computationally tractable way of Bayesian posterior sampling within our framework. Bayesian SINDy autoencoder achieves better physics discovery with lower data and fewer training epochs, along with valid uncertainty quantification suggested by the experimental studies. The Bayesian SINDy autoencoder can be applied to real video data, withaccurate physics discovery which correctly identifies the governing equation and provides a close estimate for standard physics constants like gravity g , for example, in videos of a pendulum.
Conceptor-Aided Debiasing of Contextualized Embeddings

Pre-trained language models reﬂect the inherent social biases of their training corpus. Many methods have been proposed to mitigate this issue, but they often fail to debias or they sac-riﬁce model accuracy. We use conceptors –a soft projection method–to identify and remove the bias subspace in contextual embeddings in BERT and GPT. We propose two methods of applying conceptors (1) bias subspace projection by post-processing; and (2) a new architecture, conceptor-intervened BERT (CI-BERT), which explicitly incorporates the conceptor projection into all layers during training. We ﬁnd that conceptor post-processing achieves state-of-the-art debiasing results while maintaining or improving BERT’s performance on the GLUE benchmark. Although CI-BERT’s training takes all layers’ bias into account and can outperform its post-processing counterpart in bias mitigation, CI-BERT reduces the language model accuracy. We also show the importance of carefully constructing the bias sub-space. The best results are obtained by removing outliers from the list of biased words, inter-secting them (using the conceptor AND operation), and computing their embeddings using the sentences from a cleaner corpus.
Non-centered parametric variational Bayes' approach for hierarchical inverse problems of partial differential equations

This paper proposes a non-centered parameterization based infinite-dimensional mean-field variational inference (NCP-iMFVI) approach for solving the hierarchical Bayesian inverse problems. This method can generate available estimates from the approximated posterior distribution efficiently. To avoid the mutually singular obstacle that occurred in the infinite-dimensional hierarchical approach, we propose a rigorous theory of the non-centered variational Bayesian approach. Since the non-centered parameterization weakens the connection between the parameter and the hyper-parameter, we can introduce the hyper-parameter to all terms of the eigendecomposition of the prior covariance operator. We also show the relationships between the NCP-iMFVI and infinite-dimensional hierarchical approaches with centered parameterization. The proposed algorithm is applied to three inverse problems governed by the simple smooth equation, the Helmholtz equation, and the steady-state Darcy flow equation. Numerical results confirm our theoretical findings, illustrate the efficiency of solving the iMFVI problem formulated by large-scale linear and nonlinear statistical inverse problems, and verify the mesh-independent property.
From Traditional Adaptive Data Caching to Adaptive Context Caching: A Survey

Context information is in demand more than ever with the rapid increase in the number of context-aware Internet of Things applications developed worldwide. Research in context and context-awareness is being conducted to broaden its applicability in light of many practical and technical challenges. One of the challenges is improving performance when responding to a large number of context queries. Context Management Platforms that infer and deliver context to applications measure this problem using Quality of Service (QoS) parameters. Although caching is a proven way to improve QoS, transiency of context and features such as variability and heterogeneity of context queries pose an additional real-time cost management problem. This paper presents a critical survey of the state-of-the-art in adaptive data caching with the objective of developing a body of knowledge in cost- and performance-efficient adaptive caching strategies. We comprehensively survey a large number of research publications and evaluate, compare, and contrast different techniques, policies, approaches, and schemes in adaptive caching. Our critical analysis is motivated by the focus on adaptively caching context as a core research problem. A formal definition for adaptive context caching is then proposed, followed by identified features and requirements of a well-designed, objective optimal adaptive context caching strategy.
Scalable Collaborative Learning via Representation Sharing

Privacy-preserving machine learning has become a key conundrum for multi-party artificial intelligence. Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device). In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). While relevant in several settings, both of these schemes have a high communication cost, rely on server-level computation algorithms and do not allow for tunable levels of collaboration. In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss (contrastive w.r.t. the labels). The goal is to ensure that the participants learn similar features on similar classes without sharing their input data. To do so, each client releases averaged last hidden layer activations of similar labels to a central server that only acts as a relay (i.e., is not involved in the training or aggregation of the models). Then, the clients download these last layer activations (feature representations) of the ensemble of users and distill their knowledge in their personal model using a contrastive objective. For cross-device applications (i.e., small local datasets and limited computational capacity), this approach increases the utility of the models compared to independent learning and other federated knowledge distillation (FD) schemes, is communication efficient and is scalable with the number of clients. We prove theoretically that our framework is well-posed, and we benchmark its performance against standard FD and FL on various datasets using different model architectures.
Spontaneous mutations in maize pollen are frequent in some lines and arise mainly from retrotranspositions and deletions

Significance Spontaneously arisen mutations provide the raw material for natural selection and evolution in all organisms, yet their frequency of occurrence is so low that special “mutation accumulation” lines are used to study them. In the course of conventional genetic experiments, we have found that spontaneous mutations arise with unexpectedly high frequency in the pollen of some maize lines, but not of others. Most of them result from insertions of retrotransposons present in a few copies in the genome or from deletions. Most interestingly, the mutagenic retrotransposons differ from line to line. The work reported here sheds light on the source, frequency, and nature of spontaneous mutations in maize. While studying spontaneous mutations at the maize bronze (bz) locus, we made the unexpected discovery that specific low-copy number retrotransposons are mobile in the pollen of some maize lines, but not of others. We conducted large-scale genetic experiments to isolate new bz mutations from several Bz stocks and recovered spontaneous stable mutations only in the pollen parent in reciprocal crosses. Most of the new stable bz mutations resulted from either insertions of low-copy number long terminal repeat (LTR) retrotransposons or deletions, the same two classes of mutations that predominated in a collection of spontaneous wx mutations [Wessler S (1997) The Mutants of Maize, pp 385–386]. Similar mutations were recovered at the closely linked sh locus. These events occurred with a frequency of 2–4 × 10−5 in two lines derived from W22 and in 4Co63, but not at all in B73 or Mo17, two inbreds widely represented in Corn Belt hybrids. Surprisingly, the mutagenic LTR retrotransposons differed in the active lines, suggesting differences in the autonomous element make-up of the lines studied. Some active retrotransposons, like Hopscotch, Magellan, and Bs2, a Bs1 variant, were described previously; others, like Foto and Focou in 4Co63, were not. By high-throughput sequencing of retrotransposon junctions, we established that retrotranposition of Hopscotch, Magellan, and Bs2 occurs genome-wide in the pollen of active lines, but not in the female germline or in somatic tissues. We discuss here the implications of these results, which shed light on the source, frequency, and nature of spontaneous mutations in maize.
ESTAS: Effective and Stable Trojan Attacks in Self-supervised Encoders with One Target Unlabelled Sample

Emerging self-supervised learning (SSL) has become a popular image representation encoding method to obviate the reliance on labeled data and learn rich representations from large-scale, ubiquitous unlabelled data. Then one can train a downstream classifier on top of the pre-trained SSL image encoder with few or no labeled downstream data. Although extensive works show that SSL has achieved remarkable and competitive performance on different downstream tasks, its security concerns, e.g, Trojan attacks in SSL encoders, are still not well-studied. In this work, we present a novel Trojan Attack method, denoted by ESTAS, that can enable an effective and stable attack in SSL encoders with only one target unlabeled sample. In particular, we propose consistent trigger poisoning and cascade optimization in ESTAS to improve attack efficacy and model accuracy, and eliminate the expensive target-class data sample extraction from large-scale disordered unlabelled data. Our substantial experiments on multiple datasets show that ESTAS stably achieves>99% attacks success rate (ASR) with one target-class sample. Compared to prior works, ESTAS attains>30% ASR increase and>8.3% accuracy improvement on average.
Single Stage Multi-Pose Virtual Try-On

Multi-pose virtual try-on (MPVTON) aims to fit a target garment onto a person at a target pose. Compared to traditional virtual try-on (VTON) that fits the garment but keeps the pose unchanged, MPVTON provides a better try-on experience, but is also more challenging due to the dual garment and pose editing objectives. Existing MPVTON methods adopt a pipeline comprising three disjoint modules including a target semantic layout prediction module, a coarse try-on image generator and a refinement try-on image generator. These models are trained separately, leading to sub-optimal model training and unsatisfactory results. In this paper, we propose a novel single stage model for MPVTON. Key to our model is a parallel flow estimation module that predicts the flow fields for both person and garment images conditioned on the target pose. The predicted flows are subsequently used to warp the appearance feature maps of the person and the garment images to construct a style map. The map is then used to modulate the target pose's feature map for target try-on image generation. With the parallel flow estimation design, our model can be trained end-to-end in a single stage and is more computationally efficient, resulting in new SOTA performance on existing MPVTON benchmarks. We further introduce multi-task training and demonstrate that our model can also be applied for traditional VTON and pose transfer tasks and achieve comparable performance to SOTA specialized models on both tasks.
Identifiability of Dynamic Networks: The Essential Rôle of Dources and Dinks

The article (Bazanella et al., 2019) presented the first results on generic identifiability of dynamic networks with partial excitation and partial measurements. All previous papers assumed that either all nodes are excited or all nodes are measured. One key contribution of that paper was to establish a set of necessary conditions on the excitation and measurement pattern (EMP) that guarantee generic identifiability: all sources must be excited and all sinks measured, and all other nodes must be either excited or measured. In this article, we show that two other types of nodes, which are defined by the local topology of the network, play an essential rôle in the search for a valid EMP, i.e., one that guarantees generic identifiability. We have called these nodes dources and dinks. We show that a network is generically identifiable only if, in addition to the abovementioned conditions, all dources are excited and all dinks are measured. We also show that sources and dources are the only nodes in a network that always need to be excited, and that sinks and dinks are the only nodes that need to be measured for an EMP to be valid.
When are correlations quantum?—verification and quantification of entanglement by simple measurements

The verification and quantification of experimentally created entanglement by simple measurements, especially between distant particles, is an important basic task in quantum processing. When composite systems are subjected to local measurements the measurement data will exhibit correlations, whether these systems are classical or quantum. Therefore, the observation of correlations in the classical measurement record does not automatically imply the presence of quantum correlations in the system under investigation. In this study, we explore the question of when correlations, or other measurement data, are sufficient to guarantee the existence of a certain amount of quantum correlations in the system and when additional information, such as the degree of purity of the system, is needed to do so. Various measurement settings are discussed, both numerically and analytically. Exact results and lower bounds on the least entanglement consistent with the observations are presented. The approach is suitable both for the bi-partite and the multi-partite setting.
Non-commutative intersection theory and unipotent Deligne-Milnor formula

In this paper, we prove the \emph{unipotent Deligne-Milnor formula}. Our method consists of categorifying Kato-Saito localized intersection product and then applying To\"en-Vezzosi non-commutative Chern character. In fact, a small modification of our strategy also yields Bloch conductor conjecture in several new cases. Along the way, we confirm an expectation of To\"en-Vezzosi's on the relation between their categorical intersection class and Bloch intersection number.
Continual Learning-Based MIMO Channel Estimation: A Benchmarking Study

With the proliferation of deep learning techniques for wireless communication, several works have adopted learning-based approaches to solve the channel estimation problem. While these methods are usually promoted for their computational efficiency at inference time, their use is restricted to specific stationary training settings in terms of communication system parameters, e.g., signal-to-noise ratio (SNR) and coherence time. Therefore, the performance of these learning-based solutions will degrade when the models are tested on different settings than the ones used for training. This motivates our work in which we investigate continual supervised learning (CL) to mitigate the shortcomings of the current approaches. In particular, we design a set of channel estimation tasks wherein we vary different parameters of the channel model. We focus on Gauss-Markov Rayleigh fading channel estimation to assess the impact of non-stationarity on performance in terms of the mean square error (MSE) criterion. We study a selection of state-of-the-art CL methods and we showcase empirically the importance of catastrophic forgetting in continuously evolving channel settings. Our results demonstrate that the CL algorithms can improve the interference performance in two channel estimation tasks governed by changes in the SNR level and coherence time.
Learning biological neuronal networks with artificial neural networks: neural oscillations

First-principles-based modelings have been extremely successful in providing crucial insights and predictions for complex biological functions and phenomena. However, they can be hard to build and expensive to simulate for complex living systems. On the other hand, modern data-driven methods thrive at modeling many types of high-dimensional and noisy data. Still, the training and interpretation of these data-driven models remain challenging. Here, we combine the two types of methods to model stochastic neuronal network oscillations. Specifically, we develop a class of first-principles-based artificial neural networks to provide faithful surrogates to the high-dimensional, nonlinear oscillatory dynamics produced by neural circuits in the brain. Furthermore, when the training data set is enlarged within a range of parameter choices, the artificial neural networks become generalizable to these parameters, covering cases in distinctly different dynamical regimes. In all, our work opens a new avenue for modeling complex neuronal network dynamics with artificial neural networks.
HAT: Hypergraph analysis toolbox

Recent advances in biological technologies, such as multi-way chromosome conformation capture (3C), require development of methods for analysis of multi-way interactions. Hypergraphs are mathematically tractable objects that can be utilized to precisely represent and analyze multi-way interactions. Here we present the Hypergraph Analysis Toolbox (HAT), a software package for visualization and analysis of multi-way interactions in complex systems.
Optical Isolation by Temporal Modulation: Size, Frequency, and Power Constraints

Optical isolators are indispensable components of optical networks. Magneto-optic isolators have excellent operating characteristics, including low-to-no power consumption, but are not well suited for on-chip integration. The technique of temporal modulation of dielectric constant offers an alternative way to achieve isolation without magnetic field but is not without its own drawbacks. In this work I examine diverse methods of optical isolation via temporal modulation and show that independent on whether modulation is achieved by carrier injection, Pockels and acousto-optic effects, or any other conceivable method, there is essentially the same set of constraints on footprint, modulation frequency, and, most important, on power consumption required to achieve full isolation without excessive insertion loss. This power is estimated to be on the order of at least a hundred of milliwatts and whether this requirement is acceptable will depend on ongoing progress of both magneto-optic and time modulated integrated technologies.
INJECTION OF SUPERNOVA DUST IN NEARBY PROTOPLANETARY DISKS

The early solar system contained a number of short-lived radionuclides (SLRs) such as 26Al with half-lives <15 Myr. The one-time presence of 60Fe strongly suggests that the source of these radionuclides was a nearby supernova. In this paper, we investigate the “aerogel” model, which hypothesizes that the solar system's SLRs were injected directly into the solar system's protoplanetary disk from a supernova within the same star-forming region. Previous work has shown that disks generally survive the impact of supernova ejecta, but also that little gaseous ejecta can be injected into the disk. The aerogel model hypothesizes that radionuclides in the ejecta condensed into micron-sized dust grains that were injected directly into the solar nebula disk. Here, we discuss the density structure of supernova ejecta and the observational support for dust condensation in the ejecta. We argue that supernova ejecta are clumpy and describe a model to quantify this clumpiness. We also argue that infrared observations may be underestimating the fraction of material that condenses into dust. Building on calculations of how supernova ejecta interact with protoplanetary disks, we calculate the efficiency with which dust grains in the ejecta are injected into a disk. We find that about 70% of material in grains roughly 0.4 μm in diameter can be injected into disks. If ejecta are clumpy, the solar nebula was struck by a clump with higher-than-average 26Al and 60Fe, and these elements condensed efficiently into large grains, then the abundances of SLRs in the early solar system can be explained, even if the disk lies 2 pc from the supernova explosion. The probability that all these factors are met is low, perhaps ∼10−3–10−2, and receiving as much 26Al and 60Fe as the solar system did may be a rare event. Still, the aerogel model remains a viable explanation for the origins of the radionuclides in the early solar system, and may be the most plausible one.
MEESO: A Multi-objective End-to-End Self-Optimized Approach for Automatically Building Deep Learning Models

Deep learning has been widely used in various applications from different fields such as computer vision, natural language processing, etc. However, the training models are often manually developed via many costly experiments. This manual work usually requires substantial computing resources, time, and experience. To simplify the use of deep learning and alleviate human effort, automated deep learning has emerged as a potential tool that releases the burden for both users and researchers. Generally, an automatic approach should support the diversity of model selection and the evaluation should allow users to decide upon their demands. To that end, we propose a multi-objective end-to-end self-optimized approach for constructing deep learning models automatically. Experimental results on well-known datasets such as MNIST, Fashion, and Cifar10 show that our algorithm can discover various competitive models compared with the state-of-the-art approach. In addition, our approach also introduces multi-objective trade-off solutions for both accuracy and uncertainty metrics for users to make better decisions.
Melting curve equations at high pressure

A number of equations for the melting temperature dependence on pressure, ${T}_{m}(P),$ based on the two- and one-phase approaches to melting have been obtained. All melting curves have the form ${T}_{m}=F(P)D(P),$ where $F(P)$ is the Simon (rising) melting equation and $D(P)$ is the damping function, which asymptotically slopes down under pressure. This form predicts that each solid phase has a maximum melting temperature at positive or negative pressure. The simplest equation of this form is ${T}_{m}{=T}_{0}(1+\ensuremath{\Delta}{P/a)}^{b}\mathrm{exp}(\ensuremath{-}c\ensuremath{\Delta}P),$ where $\ensuremath{\Delta}P=P\ensuremath{-}{P}_{0},$ ${P}_{0}$ is some reference pressure, and a, b, and c are parameter which are identified in terms of thermodynamic values. All melting curve data obey this equation. This implies that there exist no anomalous melting curves. All melting curves, rising, falling, and flattening, as well as curves with a maximum, are normal insofar as all of them can be described by the unified equation.