Miracle-dz/UniFAR-FTUs huggingface dataset:
A kernel-based method for data-driven koopman spectral analysis

A data-driven, kernel-based method for approximating the leading Koopman eigenvalues, eigenfunctions, and modes in problems with high-dimensional state spaces is presented. This approach uses a set of scalar observables (functions that map a state to a scalar value) that are defined implicitly by the feature map associated with a user-defined kernel function. This circumvents the computational issues that arise due to the number of functions required to span a ``sufficiently rich'' subspace of all possible scalar observables in such applications. We illustrate this method on two examples: the first is the FitzHugh-Nagumo PDE, a prototypical one-dimensional reaction-diffusion system, and the second is a set of vorticity data computed from experimentally obtained velocity data from flow past a cylinder at Reynolds number 413. In both examples, we use the output of Dynamic Mode Decomposition, which has a similar computational cost, as the benchmark for our approach.
Targeted display advertising: the case of preferential attachment

An average adult is exposed to hundreds of digital advertisements daily1, making the digital advertisement industry a classic example of a big-data-driven platform. As such, the ad-tech industry relies on historical engagement logs (clicks or purchases) to identify potentially interested users for the advertisement campaign of a partner (a seller who wants to target users for its products). The number of advertisements that are shown for a partner, and hence the historical campaign data available for a partner depends upon the budget constraints of the partner. Thus, enough data can be collected for the high-budget partners to make accurate predictions, while this is not the case with the low-budget partners. This skewed distribution of the data leads to preferential attachment of the targeted display advertising platforms towards the high-budget partners. In this paper, we develop domain-adaptation approaches to address the challenge of predicting interested users for the partners with insufficient data, i.e., the tail partners. Specifically, we develop simple yet effective approaches that leverage the similarity among the partners to transfer information from the partners with sufficient data to cold-start partners, i.e., partners without any campaign data. Our approaches readily adapt to the new campaign data by incremental fine-tuning, and hence work at varying points of a campaign, and not just the cold-start. We present an experimental analysis on the historical logs of a major display advertising platform2. Specifically, we evaluate our approaches across 149 partners, at varying points of their campaigns. Experimental results show that the proposed approaches outperform the other domain-adaptation approaches at different time points of the campaigns.
Fully nonlinear mode competition in magnetised Taylor–Couette flow

We study the nonlinear mode competition of various spiral instabilities in magnetised Taylor–Couette flow. The resulting finite-amplitude mixed-mode solution branches are tracked using the annular-parallelogram periodic domain approach developed by Deguchi & Altmeyer (Phys. Rev. E, vol. 87, 2013, 043017). Mode competition phenomena are studied in both the anticyclonic and cyclonic Rayleigh-stable regimes. In the anticyclonic sub-rotation regime, with the inner cylinder rotating faster than the outer, Hollerbach et al. (Phys. Rev. Lett., vol. 104, 2010, 044502) found competing axisymmetric and non-axisymmetric magneto-rotational linearly unstable modes within the parameter range where experimental investigation is feasible. Here we confirm the existence of mode competition and compute the nonlinear mixed-mode solutions that result from it. In the cyclonic super-rotating regime, with the inner cylinder rotating slower than the outer, Deguchi (Phys. Rev. E, vol. 95, 2017, 021102) recently found a non-axisymmetric purely hydrodynamic linear instability that coexists with the non-axisymmetric magneto-rotational instability discovered a little earlier by Rüdiger et al. (Phys. Fluids, vol. 28, 2016, 014105). We show that nonlinear interactions of these instabilities give rise to rich pattern-formation phenomena leading to drastic angular momentum transport enhancement/reduction.
Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources

Many tasks in robot-assisted surgeries (RAS) can be represented by finite-state machines (FSMs), where each state represents either an action (such as picking up a needle) or an observation (such as bleeding). A crucial step towards the automation of such surgical tasks is the temporal perception of the current surgical scene, which requires a real-time estimation of the states in the FSMs. The objective of this work is to estimate the current state of the surgical task based on the actions performed or events occurred as the task progresses. We propose Fusion-KVE, a unified surgical state estimation model that incorporates multiple data sources including the Kinematics, Vision, and system Events. Additionally, we examine the strengths and weaknesses of different state estimation models in segmenting states with different representative features or levels of granularity. We evaluate our model on the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), as well as a more complex dataset involving robotic intra-operative ultrasound (RIOUS) imaging, created using the da Vinci® Xi surgical system. Our model achieves a superior frame-wise state estimation accuracy up to 89.4%, which improves the state-of-the-art surgical state estimation models in both JIGSAWS suturing dataset and our RIOUS dataset.
On abstraction-based controller design with output feedback

We consider abstraction-based design of output-feedback controllers for dynamical systems with a finite set of inputs and outputs against specifications in linear-time temporal logic. The usual procedure for abstraction-based controller design (ABCD) first constructs a finite-state abstraction of the underlying dynamical system, and second, uses reactive synthesis techniques to compute an abstract state-feedback controller on the abstraction. In this context, our contribution is two-fold: (I) we define a suitable relation between the original system and its abstraction which characterizes the soundness and completeness conditions for an abstract state-feedback controller to be refined to a concrete output-feedback controller for the original system, and (II) we provide an algorithm to compute a sound finite-state abstraction fulfilling this relation. Our relation generalizes feedback-refinement relations from ABCD with state-feedback. Our algorithm for constructing sound finite-state abstractions is inspired by the simultaneous reachability and bisimulation minimization algorithm of Lee and Yannakakis. We lift their idea to the computation of an observation-equivalent system and show how sound abstractions can be obtained by stopping this algorithm at any point. Additionally, our new algorithm produces a realization of the topological closure of the input/output behavior of the original system if it is finite-state realizable.
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing

In this paper, we propose a novel model compression approach to effectively compress BERT by progressive module replacing. Our approach first divides the original BERT into several modules and builds their compact substitutes. Then, we randomly replace the original modules with their substitutes to train the compact modules to mimic the behavior of the original modules. We progressively increase the probability of replacement through the training. In this way, our approach brings a deeper level of interaction between the original and compact models. Compared to the previous knowledge distillation approaches for BERT compression, our approach does not introduce any additional loss function. Our approach outperforms existing knowledge distillation approaches on GLUE benchmark, showing a new perspective of model compression.
DynamicPPL: Stan-like Speed for Dynamic Probabilistic Models

We present the preliminary high-level design and features of DynamicPPL.jl, a modular library providing a lightning-fast infrastructure for probabilistic programming. Besides a computational performance that is often close to or better than Stan, DynamicPPL provides an intuitive DSL that allows the rapid development of complex dynamic probabilistic programs. Being entirely written in Julia, a high-level dynamic programming language for numerical computing, DynamicPPL inherits a rich set of features available through the Julia ecosystem. Since DynamicPPL is a modular, stand-alone library, any probabilistic programming system written in Julia, such as Turing.jl, can use DynamicPPL to specify models and trace their model parameters. The main features of DynamicPPL are: 1) a meta-programming based DSL for specifying dynamic models using an intuitive tilde-based notation; 2) a tracing data-structure for tracking RVs in dynamic probabilistic models; 3) a rich contextual dispatch system allowing tailored behaviour during model execution; and 4) a user-friendly syntax for probabilistic queries. Finally, we show in a variety of experiments that DynamicPPL, in combination with Turing.jl, achieves computational performance that is often close to or better than Stan.
Data augmentation with Mobius transformations

Data augmentation has led to substantial improvements in the performance and generalization of deep models, and remains a highly adaptable method to evolving model architectures and varying amounts of data—in particular, extremely scarce amounts of available training data. In this paper, we present a novel method of applying Möbius transformations to augment input images during training. Möbius transformations are bijective conformal maps that generalize image translation to operate over complex inversion in pixel space. As a result, Möbius transformations can operate on the sample level and preserve data labels. We show that the inclusion of Möbius transformations during training enables improved generalization over prior sample-level data augmentation techniques such as cutout and standard crop-and-flip transformations, most notably in low data regimes.
Cosmic-Ray Driven Outflows to Mpc Scales from L* Galaxies

We study the effects of cosmic rays (CRs) on outflows from star-forming galaxies in the circum and inter-galactic medium (CGM/IGM), in high-resolution, fully-cosmological FIRE-2 simulations (accounting for mechanical and radiative stellar feedback, magnetic fields, anisotropic conduction/viscosity/CR diffusion and streaming, and CR losses). We showed previously that massive (Mhalo ≳ 1011 M⊙), low-redshift (z ≲ 1 − 2) halos can have CR pressure dominate over thermal CGM pressure and balance gravity, giving rise to a cooler CGM with an equilibrium density profile. This dramatically alters outflows. Absent CRs, high gas thermal pressure in massive halos “traps” galactic outflows near the disk, so they recycle. With CRs injected in supernovae as modeled here, the low-pressure halo allows “escape” and CR pressure gradients continuously accelerate this material well into the IGM in “fast” outflows, while lower-density gas at large radii is accelerated in-situ into “slow” outflows that extend to >Mpc scales. CGM/IGM outflow morphologies are radically altered: they become mostly volume-filling (with inflow in a thin mid-plane layer) and coherently biconical from the disk to >Mpc. The CR-driven outflows are primarily cool (T ∼ 105 K) and low-velocity. All of these effects weaken and eventually vanish at lower halo masses (≲ 1011 M⊙) or higher redshifts (z ≳ 1 − 2), reflecting the ratio of CR to thermal+gravitational pressure in the outer halo. We present a simple analytic model which explains all of the above phenomena. We caution that these predictions may depend on uncertain CR transport physics.
Grammar Filtering For Syntax-Guided Synthesis

Programming-by-example (PBE) is a synthesis paradigm that allows users to generate functions by simply providing input-output examples. While a promising interaction paradigm, synthesis is still too slow for realtime interaction and more widespread adoption. Existing approaches to PBE synthesis have used automated reasoning tools, such as SMT solvers, as well as works applying machine learning techniques. At its core, the automated reasoning approach relies on highly domain specific knowledge of programming languages. On the other hand, the machine learning approaches utilize the fact that when working with program code, it is possible to generate arbitrarily large training datasets. In this work, we propose a system for using machine learning in tandem with automated reasoning techniques to solve Syntax Guided Synthesis (SyGuS) style PBE problems. By preprocessing SyGuS PBE problems with a neural network, we can use a data driven approach to reduce the size of the search space, then allow automated reasoning-based solvers to more quickly find a solution analytically. Our system is able to run atop existing SyGuS PBE synthesis tools, decreasing the runtime of the winner of the 2019 SyGuS Competition for the PBE Strings track by 47.65% to outperform all of the competing tools.
Learning Class Regularized Features for Action Recognition

Training Deep Convolutional Neural Networks (CNNs) is based on the notion of using multiple kernels and non-linearities in their subsequent activations to extract useful features. The kernels are used as general feature extractors without specific correspondence to the target class. As a result, the extracted features do not correspond to specific classes. Subtle differences between similar classes are modeled in the same way as large differences between dissimilar classes. To overcome the class-agnostic use of kernels in CNNs, we introduce a novel method named Class Regularization that performs class-based regularization of layer activations. We demonstrate that this not only improves feature search during training, but also allows an explicit assignment of features per class during each stage of the feature extraction process. We show that using Class Regularization blocks in state-of-the-art CNN architectures for action recognition leads to systematic improvement gains of 1.8%, 1.2% and 1.4% on the Kinetics, UCF-101 and HMDB-51 datasets, respectively.
A learning without forgetting approach to incorporate artifact knowledge in polyp localization tasks

Colorectal polyps are abnormalities in the colon tissue that can develop into colorectal cancer. The survival rate for patients is higher when the disease is detected at an early stage and polyps can be removed before they develop into malignant tumors. Deep learning methods have become the state of art in automatic polyp detection. However, the performance of current models heavily relies on the size and quality of the training datasets. Endoscopic video sequences tend to be corrupted by different artifacts affecting visibility and hence, the detection rates. In this work, we analyze the effects that artifacts have in the polyp localization problem. For this, we evaluate the RetinaNet architecture, originally defined for object localization. We also define a model inspired by the learning without forgetting framework, which allows us to employ artifact detection knowledge in the polyp localization problem. Finally, we perform several experiments to analyze the influence of the artifacts in the performance of these models. To our best knowledge, this is the first extensive analysis of the influence of artifact in polyp localization and the first work incorporating learning without forgetting ideas for simultaneous artifact and polyp localization tasks.
MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing

Personal mobile sensing is fast permeating our daily lives to enable activity monitoring, healthcare and rehabilitation. Combined with deep learning, these applications have achieved significant success in recent years. Different from conventional cloud-based paradigms, running deep learning on devices offers several advantages including data privacy preservation and low-latency response for both model inference and update. Since data collection is costly in reality, Google’s Federated Learning offers not only complete data privacy but also better model robustness based on multiple user data. However, personal mobile sensing applications are mostly user-specific and highly affected by environment. As a result, continuous local changes may seriously affect the performance of a global model generated by Federated Learning. In addition, deploying Federated Learning on a local server, e.g., edge server, may quickly reach the bottleneck due to resource constraint and serious failure by attacks. Towards pushing deep learning on devices, we present MDLdroid, a novel decentralized mobile deep learning framework to enable resource-aware on-device collaborative learning for personal mobile sensing applications. To address resource limitation, we propose a ChainSGD-reduce approach which includes a novel chain-directed Synchronous Stochastic Gradient Descent algorithm to effectively reduce overhead among multiple devices. We also design an agent-based multi-goal reinforcement learning mechanism to balance resources in a fair and efficient manner. Our evaluations show that our model training on off-the-shelf mobile devices achieves 2x to 3.5x faster than single-device training, and 1.5x faster than the master-slave approach.
Inference of Polygenic Factors Associated with Breast Cancer Gene Interaction Networks from Discrete Data Utilizing Poisson Multivariate Mutual Information.

In this work we introduce a new methodology to infer from gene expression data the complex interactions associated with polygenetic diseases that remain a major frontier in understanding factors in human health. In many cases disease may be related to the covariance of several genes, rather than simply the variance of a single gene, making network inference crucial to the development of potential treatments. Specifically we investigate the network of factors and associations involved in developing breast cancer from gene expression data. Our approach is information theoretic, but a major obstacle has been the discrete nature of such data that is well described as a multi-variate Poisson process. In fact despite that mutual information is generally a well regarded approach for developing networks of association in data science of complex systems across many disciplines, until now a good method to accurately and efficiently compute entropies from such processes as been lacking. Nonparameteric methods such as the popular k-nearest neighbors (KNN) methods are slow converging and thus require unrealistic amounts of data. We will use the causation entropy (CSE) principle, together with the associated greedy search algorithm optimal CSE (oCSE) as a network inference method to deduce the actual structure, with our multi-variate Poisson estimator developed here as the core computational engine. We show that the Poisson version of oCSE outperforms both the Kraskov-Stogbauer-Grassberger (KSG) oCSE method (which is a KNN method for estimating the entropy) and the Gaussian oCSE method on synthetic data. We present the results for a breast cancer gene expression data set.
Interpolation under latent factor regression models

This work studies finite-sample properties of the risk of the minimum-norm interpolating predictor in high-dimensional regression models. If the effective rank of the covariance matrix $\Sigma$ of the $p$ regression features is much larger than the sample size $n$, we show that the min-norm interpolating predictor is not desirable, as its risk approaches the risk of trivially predicting the response by $0$. However, our detailed finite sample analysis reveals, surprisingly, that this behavior is not present when the regression response and the features are jointly low-dimensional, and follow a widely used factor regression model. Within this popular model class, and when the effective rank of $\Sigma$ is smaller than $n$, while still allowing for $p \gg n$, both the bias and the variance terms of the excess risk can be controlled, and the risk of the minimum-norm interpolating predictor approaches optimal benchmarks. Moreover, through a detailed analysis of the bias term, we exhibit model classes under which our upper bound on the excess risk approaches zero, while the corresponding upper bound in the recent work arXiv:1906.11300v3 diverges. Furthermore, we show that minimum-norm interpolating predictors analyzed under factor regression models, despite being model-agnostic, can have similar risk to model-assisted predictors based on principal components regression, in the high-dimensional regime.
Impact of the Interaction Network on the Dynamics of Word-of-Mouth with Information Seeking

Word-of-Mouth refers to the dynamics of interpersonal communication occurring during the diffusion of innovations (novel practices, ideas or products). According to field studies, word-of-mouth is made of both information seeking and proactive communication: individuals first become aware of the existence of an innovation, then start actively seeking out for the expert knowledge required to evaluate the innovation; when they hold the expert knowledge, they might start promoting it pro-actively. Successful diffusion of innovation requires the individuals to hold both awareness and expert knowledge, so they can evaluate the innovation and use it properly. A computational model "USA/IPK" was recently proposed to study the role and impact of information seeking on the dynamics of word-of-mouth. We propose here an analysis of the impact of the network of interaction on the dynamics of this model. We compare the dynamics of the model over networks generated with different algorithms with the original dynamics. The results demonstrate the dynamics of the model are similar across tested networks, with the noticeable exception of the efficiency of the diffusion which varies between networks having similar densities and sizes.
Exciton 
g
 factors of van der Waals heterostructures from first-principles calculations

Interlayer excitons present in van der Waals heterostructures of transition metal dichalcogenides exhibit remarkably distinct effective $g$ factors compared to their monolayer counterparts. Here, the authors employ first-principles calculations to investigate the interlayer exciton $g$ factors in MoSe${}_{2}$/WSe${}_{2}$ vdW heterostructures revealing that they are strongly spin and stacking dependent. Furthermore, within the present approach it is possible to unambiguously identify the nature of various previously measured interlayer exciton peaks. This first-principles approach can potentially be applied to calculate exciton $g$ factors of any semiconductor.