Miracle-dz/UniFAR-FTUs huggingface dataset:
Trust-Region Approximation of Extreme Trajectories in Power System Dynamics

In this work we present a novel technique, based on a trust-region optimization algorithm and second-order trajectory sensitivities, to compute the extreme trajectories of power system dynamic simulations given a bounded set that represents parametric uncertainty. We show how this method, while remaining computationally efficient compared with sampling-based techniques, overcomes the limitations of previous sensitivity-based techniques to approximate the bounds of the trajectories when the local approximation loses validity because of the nonlinearity. We present several numerical experiments that showcase the accuracy and scalability of the technique, including a demonstration on the IEEE New England test system.
Existence of gravity-capillary Crapper waves with concentrated vorticity

The aim of this paper is to prove the existence of gravity-capillary Crapper waves with the presence of vorticity. In particular, we consider a concentrated vorticity: point vortex and vortex patch. We show that for small gravity and small vorticity it is possible to demonstrate that the waves are overhanging.
Verifix: Verified Repair of Programming Assignments

Automated feedback generation for introductory programming assignments is useful for programming education. Most works try to generate feedback to correct a student program by comparing its behavior with an instructor’s reference program on selected tests. In this work, our aim is to generate verifiably correct program repairs as student feedback. A student-submitted program is aligned and composed with a reference solution in terms of control flow, and the variables of the two programs are automatically aligned via predicates describing the relationship between the variables. When verification attempt for the obtained aligned program fails, we turn a verification problem into a MaxSMT problem whose solution leads to a minimal repair. We have conducted experiments on student assignments curated from a widely deployed intelligent tutoring system. Our results show that generating verified repair without sacrificing the overall repair rate is possible. In fact, our implementation, Verifix, is shown to outperform Clara, a state-of-the-art tool, in terms of repair rate. This shows the promise of using verified repair to generate high confidence feedback in programming pedagogy settings.
Deep Linear Networks Dynamics: Low-Rank Biases Induced by Initialization Scale and L2 Regularization

For deep linear networks (DLN), various hyperparameters alter the dynamics of training dramatically. We investigate how the rank of the linear map found by gradient descent is affected by (1) the initialization norm and (2) the addition of L 2 regularization on the parameters. For (1), we study two regimes: (1a) the linear/lazy regime, for large norm initialization; (1b) a "saddle-to-saddle" regime for small initialization norm. In the (1a) setting, the dynamics of a DLN of any depth is similar to that of a standard linear model, without any low-rank bias. In the (1b) setting, we conjecture that throughout training, gradient descent approaches a sequence of saddles, each corresponding to linear maps of increasing rank, until reaching a minimal rank global minimum. We support this conjecture with a partial proof and some numerical experiments. For (2), we show that adding a L 2 regularization on the parameters corresponds to the addition to the cost of a L p -Schatten (quasi)norm on the linear map with p = 2 L (for a depth-L network), leading to a stronger low-rank bias as L grows. The effect of L 2 regularization on the loss surface depends on the depth: for shallow networks, all critical points are either strict saddles or global minima, whereas for deep networks, some local minima appear. We numerically observe that these local minima can generalize better than global ones in some settings.
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA

In this paper, we introduce ELECTRA-style tasks to cross-lingual language model pre-training. Specifically, we present two pre-training tasks, namely multilingual replaced token detection, and translation replaced token detection. Besides, we pretrain the model, named as XLM-E, on both multilingual and parallel corpora. Our model outperforms the baseline models on various cross-lingual understanding tasks with much less computation cost. Moreover, analysis shows that XLM-E tends to obtain better cross-lingual transferability.
On the value of a general stochastic differential game with ergodic payoff

In this paper we study a type of zero-sum stochastic differential games with ergodic payoff in which the diffusion system does not need to be non-degenerate. We first show the existence of a viscosity solution of the associated ergodic Hamilton-Jacobi-Bellman-Isaacs equation under the dissipativity condition. With the help of this viscosity solution, we then give the estimates for the upper and lower ergodic value functions by constructing a series of non-degenerate approximating processes and using the supand inf-convolution techniques. Finally, we obtain the existence of a value under the Isaacs condition and its representation formulae. In addition, we apply our results to study a type of pollution accumulation problems with the long-run average social welfare.
Learning Bounds for Open-Set Learning

Traditional supervised learning aims to train a classifier in the closed-set world, where training and test samples share the same label space. In this paper, we target a more challenging and realistic setting: open-set learning (OSL), where there exist test samples from the classes that are unseen during training. Although researchers have designed many methods from the algorithmic perspectives, there are few methods that provide generalization guarantees on their ability to achieve consistent performance on different training samples drawn from the same distribution. Motivated by the transfer learning and probably approximate correct (PAC) theory, we make a bold attempt to study OSL by proving its generalization error-given training samples with size n, the estimation error will get close to order O_p(1/\sqrt{n}). This is the first study to provide a generalization bound for OSL, which we do by theoretically investigating the risk of the target classifier on unknown classes. According to our theory, a novel algorithm, called auxiliary open-set risk (AOSR) is proposed to address the OSL problem. Experiments verify the efficacy of AOSR. The code is available at github.com/Anjin-Liu/Openset_Learning_AOSR.
A Magnetospheric Dichotomy for Pulsars with Extreme Inclinations

In this work, we expand on a comment by Lyne et al. (2017), that intermittent pulsars tend to congregate near a stripe in the logarithmic period versus period-derivative diagram. Such a stripe represents a small range of polar cap electric potential. Taking into account also the fact (already apparent in their Figure 7, but not explicitly stated there) that high-fraction nulling pulsars also tend to reside within this and an additional stripe, we make the observation that the two stripes further match the “death lines” for double- and single-pole interpulses, associated with nearly orthogonal and aligned rotators, respectively. These extreme inclinations are known to suffer from pair production deficiencies, so we propose to explain intermittency and high-fraction nulling by reinvigorating some older quiescent (no pulsar wind or radio emission) “electrosphere” solutions. Specifically, as the polar potential drops below the two threshold bands (i.e., the two stripes), corresponding to the aligned and orthogonal rotators, their respective magnetospheres transition from being of the active pair-production-sustained-type into becoming the electrospheres, in which charges are only lifted from the star. The borderline cases sitting in the gap outside of the stable regime of either case manifest as high-fraction nullers. Hall evolution of the magnetic field inside orthogonally rotating neutron stars can furthermore drive secular regime changes, resulting in intermittent pulsars.
Switchover phenomenon induced by epidemic seeding on geometric networks

Significance Initial seeding of an epidemic from the best-connected nodes of a network would intuitively lead to the largest outbreak. We challenge this picture and explore a switchover phenomenon: Epidemics started from the central part of a geometric metapopulation network can reach more individuals only if the basic reproduction number is small, but if the epidemic is more infectious, it reaches a larger population when seeded from uniformly selected nodes. We identify spatial geometry to amplify this effect in both data-driven and synthetic epidemic models, and we give rigorous proofs that this phenomenon appears in various random networks. Our results help us understand why real epidemics started from seemingly similar conditions may have significantly different outcomes. It is a fundamental question in disease modeling how the initial seeding of an epidemic, spreading over a network, determines its final outcome. One important goal has been to find the seed configuration, which infects the most individuals. Although the identified optimal configurations give insight into how the initial state affects the outcome of an epidemic, they are unlikely to occur in real life. In this paper we identify two important seeding scenarios, both motivated by historical data, that reveal a complex phenomenon. In one scenario, the seeds are concentrated on the central nodes of a network, while in the second one, they are spread uniformly in the population. Comparing the final size of the epidemic started from these two initial conditions through data-driven and synthetic simulations on real and modeled geometric metapopulation networks, we find evidence for a switchover phenomenon: When the basic reproduction number R0 is close to its critical value, more individuals become infected in the first seeding scenario, but for larger values of R0, the second scenario is more dangerous. We find that the switchover phenomenon is amplified by the geometric nature of the underlying network and confirm our results via mathematically rigorous proofs, by mapping the network epidemic processes to bond percolation. Our results expand on the previous finding that, in the case of a single seed, the first scenario is always more dangerous and further our understanding of why the sizes of consecutive waves of a pandemic can differ even if their epidemic characters are similar.
Multi-Source domain adaptation via supervised contrastive learning and confident consistency regularization

Multi-Source Unsupervised Domain Adaptation (multi-source UDA) aims to learn a model from several labeled source domains while performing well on a different target domain where only unlabeled data are available at training time. To align source and target features distributions, several recent works use source and target explicit statistics matching such as features moments or class centroids. Yet, these approaches do not guarantee class conditional distributions alignment across domains. In this work, we propose a new framework called Contrastive Multi-Source Domain Adaptation (CMSDA) for multi-source UDA that addresses this limitation. Discriminative features are learned from interpolated source examples via cross entropy minimization and from target examples via consistency regularization and hard pseudo-labeling. Simultaneously, interpolated source examples are leveraged to align source class conditional distributions through an interpolated version of the supervised contrastive loss. This alignment leads to more general and transferable features which further improves the generalization on the target domain. Extensive experiments have been carried out on three standard multi-source UDA datasets where our method reports state-of-the-art results.
Fixed points of monotonic and (weakly) scalable neural networks

—We derive conditions for the existence of ﬁxed points of neural networks, an important research objective to understand their behavior in modern applications involving autoencoders and loop unrolling techniques, among others. In particular, we focus on networks with nonnegative inputs and nonnegative network parameters, as often considered in the literature. We show that such networks can be recognized as monotonic and (weakly) scalable functions within the framework of nonlinear Perron-Frobenius theory. This fact enables us to derive conditions for the existence of a nonempty ﬁxed point set of the neural networks, and these conditions are weaker than those obtained recently using arguments in convex analysis, which are typically based on the assumption of nonexpansivity of the activation functions. Furthermore, we prove that the shape of the ﬁxed point set of monotonic and weakly scalable neural networks is often an interval, which degenerates to a point for the case of scalable networks. The chief results of this paper are veriﬁed in numerical simulations, where we consider an autoencoder-type network that ﬁrst compresses angular power spectra in massive MIMO systems, and, second, reconstruct the input spectra from the compressed signal.
Recent Advances in Fibrosis and Scar Segmentation From Cardiac MRI: A State-of-the-Art Review and Future Perspectives

Segmentation of cardiac fibrosis and scars is essential for clinical diagnosis and can provide invaluable guidance for the treatment of cardiac diseases. Late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) has been successful in guiding the clinical diagnosis and treatment reliably. For LGE CMR, many methods have demonstrated success in accurately segmenting scarring regions. Co-registration with other non-contrast-agent (non-CA) modalities [e.g., balanced steady-state free precession (bSSFP) cine magnetic resonance imaging (MRI)] can further enhance the efficacy of automated segmentation of cardiac anatomies. Many conventional methods have been proposed to provide automated or semi-automated segmentation of scars. With the development of deep learning in recent years, we can also see more advanced methods that are more efficient in providing more accurate segmentations. This paper conducts a state-of-the-art review of conventional and current state-of-the-art approaches utilizing different modalities for accurate cardiac fibrosis and scar segmentation.
Phytosterols from the Rice (Oryza sativa) Bran

Three phytosterols of rare occurrence, schleicheol 2 (1), -hydroxysitosterol (2), and -hydroxysitosterol (3), were isolated from the n-hexane fraction of rice (Oryza sativa) bran, for the first time. Some nuclear magnetic resonance (NMR) assignments in the literatures are inaccurate. This study employed two-dimensional NMR experiments to identify exact peak assignments.
Entanglement order parameters and critical behavior for topological phase transitions and beyond

Topological phases are exotic quantum phases which are lacking the characterization in terms of order parameters. In this paper, we develop a unified framework based on variational iPEPS for the quantitative study of both topological and conventional phase transitions through entanglement order parameters. To this end, we employ tensor networks with suitable physical and/or entanglement symmetries encoded, and allow for order parameters detecting the behavior of any of those symmetries, both physical and entanglement ones. First, this gives rise to entanglement-based order parameters for topological phases. These topological order parameters allow to quantitatively probe topological phase transitions and to identify their universal behavior. We apply our framework to the study of the Toric Code model in different magnetic fields, which in some cases maps to the (2+1)D Ising model. We identify 3D Ising critical exponents for the entire transition, consistent with those special cases and general belief. However, we moreover find an unknown critical exponent beta=0.021. We then apply our framework of entanglement order parameters to conventional phase transitions. We construct a novel type of disorder operator (or disorder parameter), which is non-zero in the disordered phase and measures the response of the wavefunction to a symmetry twist in the entanglement. We numerically evaluate this disorder operator for the (2+1)D transverse field Ising model, where we again recover a critical exponent hitherto unknown in the model, beta=0.024, consistent with the findings for the Toric Code. This shows that entanglement order parameters can provide additional means of characterizing the universal data both at topological and conventional phase transitions, and altogether demonstrates the power of this framework to identify the universal data underlying the transition.
COKE: Communication-Censored Decentralized Kernel Learning

This paper studies the decentralized optimization and learning problem where multiple interconnected agents aim to learn an optimal decision function defined over a reproducing kernel Hilbert space by jointly minimizing a global objective function, with access to their own locally observed dataset. As a non-parametric approach, kernel learning faces a major challenge in distributed implementation: the decision variables of local objective functions are data-dependent and thus cannot be optimized under the decentralized consensus framework without any raw data exchange among agents. To circumvent this major challenge, we leverage the random feature (RF) approximation approach to enable consensus on the function modeled in the RF space by data-independent parameters across different agents. We then design an iterative algorithm, termed DKLA, for fast-convergent implementation via ADMM. Based on DKLA, we further develop a communication-censored kernel learning (COKE) algorithm that reduces the communication load of DKLA by preventing an agent from transmitting at every iteration unless its local updates are deemed informative. Theoretical results in terms of linear convergence guarantee and generalization performance analysis of DKLA and COKE are provided. Comprehensive tests on both synthetic and real datasets are conducted to verify the communication efficiency and learning effectiveness of COKE.
Koopman Spectrum Nonlinear Regulator and Provably Efficient Online Learning

Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, behaviors with sudden accelerations that waste energy and lack predictability. In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics. This induces a broader class of dynamical behaviors that evolve over stable manifolds such as nonlinear oscillators, closed loops, and smooth movements. We demonstrate that some dynamics realizations that are not possible with a cumulative cost are feasible in this paradigm. Moreover, we present a provably efficient online learning algorithm for our problem that enjoys a sub-linear regret bound under some structural assumptions.
The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning

Although machine learning models typically experience a drop in performance on out-of-distribution data, accuracies on in- versus out-of-distribution data are widely observed to follow a single linear trend when evaluated across a testbed of models. Models that are more accurate on the out-of-distribution data relative to this baseline exhibit"effective robustness"and are exceedingly rare. Identifying such models, and understanding their properties, is key to improving out-of-distribution performance. We conduct a thorough empirical investigation of effective robustness during fine-tuning and surprisingly find that models pre-trained on larger datasets exhibit effective robustness during training that vanishes at convergence. We study how properties of the data influence effective robustness, and we show that it increases with the larger size, more diversity, and higher example difficulty of the dataset. We also find that models that display effective robustness are able to correctly classify 10% of the examples that no other current testbed model gets correct. Finally, we discuss several strategies for scaling effective robustness to the high-accuracy regime to improve the out-of-distribution accuracy of state-of-the-art models.
Asymptotics in the Dirichlet Problem for Second Order Elliptic Equations with Degeneration on the Boundary

We study small perturbations of the Dirichlet problems for second order elliptic equations that degenerate on the boundary. The limit of the solution, as the perturbation tends to zero, is calculated. The result is based on a certain asymptotic self-similarity near the boundary, which holds in the generic case. In the last section, we briefly consider the stabilization of solutions to the corresponding parabolic equations with a small parameter. Metastability effects arise in this case: the asymptotics of the solution depends on the time scale. Initial-boundary value problem with the Neumann boundary condition is discussed in the last section as well. 2010 Mathematics Subject Classification Numbers: 35B40, 35J25, 60J60, 60F10.
A finite atlas for solution manifolds of differential systems with discrete state-dependent delays

Let r > 0, n ∈ N,k ∈ N. Consider the delay differential equation x(t) = g(x(t− d1(Lxt)), . , x(t− dk(Lxt))) for g : (R) ⊃ V → R continuously differentiable, L a continuous linear map from C([−r, 0],R) into a finite-dimensional vectorspace F , each dk : F ⊃ W → [0, r], k = 1, . ,k, continuously differentiable, and xt(s) = x(t + s). The solutions define a semiflow of continuously differentiable solution operators on the submanifold Xf ⊂ C([−r, 0],R) which is given by the compatibility condition φ′(0) = f(φ) with f(φ) = g(φ(−d1(Lφ)), . , φ(−dk(Lφ))). We prove that Xf has a finite atlas of at most 2 k manifold charts, whose domains are almost graphs over X0. The size of the atlas depends solely on the zerosets of the delay functions dk.
DF-Conformer: Integrated Architecture of Conv-Tasnet and Conformer Using Linear Complexity Self-Attention for Speech Enhancement

Single-channel speech enhancement (SE) is an important task in speech processing. A widely used framework combines an anal-ysis/synthesis filterbank with a mask prediction network, such as the Conv-TasNet architecture. In such systems, the denoising performance and computational efficiency are mainly affected by the structure of the mask prediction network. In this study, we aim to improve the sequential modeling ability of Conv-TasNet architectures by integrating Conformer layers into a new mask prediction network. To make the model computationally feasible, we extend the Conformer using linear complexity attention and stacked 1-D dilated depthwise convolution layers. We trained the model on 3,396 hours of noisy speech data, and show that (i) the use of linear complexity attention avoids high computational complexity, and (ii) our model achieves higher scale-invariant signal-to-noise ratio than the improved time-dilated convolution network (TDCN++), an extended version of Conv-TasNet.