Miracle-dz/UniFAR-FTUs huggingface dataset:
How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?

A recent line of research on deep learning focuses on the extremely over-parameterized setting, and shows that when the network width is larger than a high degree polynomial of the training sample size $n$ and the inverse of the target accuracy $\epsilon^{-1}$, deep neural networks learned by (stochastic) gradient descent enjoy nice optimization and generalization guarantees. Very recently, it is shown that under certain margin assumption on the training data, a polylogarithmic width condition suffices for two-layer ReLU networks to converge and generalize (Ji and Telgarsky, 2019). However, how much over-parameterization is sufficient to guarantee optimization and generalization for deep neural networks still remains an open question. In this work, we establish sharp optimization and generalization guarantees for deep ReLU networks. Under various assumptions made in previous work, our optimization and generalization guarantees hold with network width polylogarithmic in $n$ and $\epsilon^{-1}$. Our results push the study of over-parameterized deep neural networks towards more practical settings.
In-Vitro MPI-guided IVOCT catheter tracking in real time for motion artifact compensation

Purpose Using 4D magnetic particle imaging (MPI), intravascular optical coherence tomography (IVOCT) catheters are tracked in real time in order to compensate for image artifacts related to relative motion. Our approach demonstrates the feasibility for bimodal IVOCT and MPI in-vitro experiments. Material and methods During IVOCT imaging of a stenosis phantom the catheter is tracked using MPI. A 4D trajectory of the catheter tip is determined from the MPI data using center of mass sub-voxel strategies. A custom built IVOCT imaging adapter is used to perform different catheter motion profiles: no motion artifacts, motion artifacts due to catheter bending, and heart beat motion artifacts. Two IVOCT volume reconstruction methods are compared qualitatively and quantitatively using the DICE metric and the known stenosis length. Results The MPI-tracked trajectory of the IVOCT catheter is validated in multiple repeated measurements calculating the absolute mean error and standard deviation. Both volume reconstruction methods are compared and analyzed whether they are capable of compensating the motion artifacts. The novel approach of MPI-guided catheter tracking corrects motion artifacts leading to a DICE coefficient with a minimum of 86% in comparison to 58% for a standard reconstruction approach. Conclusions IVOCT catheter tracking with MPI in real time is an auspicious method for radiation free MPI-guided IVOCT interventions. The combination of MPI and IVOCT can help to reduce motion artifacts due to catheter bending and heart beat for optimized IVOCT volume reconstructions.
Decision Propagation Networks for Image Classification

High-level (e.g., semantic) features encoded in the latter layers of convolutional neural networks are extensively exploited for image classification, leaving low-level (e.g., color) features in the early layers underexplored. In this paper, we propose a novel Decision Propagation Module (DPM) to make an intermediate decision that could act as category-coherent guidance extracted from early layers, and then propagate it to the latter layers. Therefore, by stacking a collection of DPMs into a classification network, the generated Decision Propagation Network is explicitly formulated as to progressively encode more discriminative features guided by the decision, and then refine the decision based on the new generated features layer by layer. Comprehensive results on four publicly available datasets validate DPM could bring significant improvements for existing classification networks with minimal additional computational cost and is superior to the state-of-the-art methods.
Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing

Human parsing, or human body part semantic segmentation, has been an active research topic due to its wide potential applications. In this paper, we propose a novel GRAph PYramid Mutual Learning (Grapy-ML) method to address the cross-dataset human parsing problem, where the annotations are at different granularities. Starting from the prior knowledge of the human body hierarchical structure, we devise a graph pyramid module (GPM) by stacking three levels of graph structures from coarse granularity to fine granularity subsequently. At each level, GPM utilizes the self-attention mechanism to model the correlations between context nodes. Then, it adopts a top-down mechanism to progressively refine the hierarchical features through all the levels. GPM also enables efficient mutual learning. Specifically, the network weights of the first two levels are shared to exchange the learned coarse-granularity information across different datasets. By making use of the multi-granularity labels, Grapy-ML learns a more discriminative feature representation and achieves state-of-the-art performance, which is demonstrated by extensive experiments on the three popular benchmarks, e.g. CIHP dataset. The source code is publicly available at https://github.com/Charleshhy/Grapy-ML.
High-Precision Nuclear Forces From Chiral EFT: State-of-the-Art, Challenges, and Outlook

We review a new generation of nuclear forces derived in chiral effective field theory using the recently proposed semilocal regularization method. We outline the conceptual foundations of nuclear chiral effective field theory, discuss all steps needed to compute nuclear observables starting from the effective chiral Lagrangian and consider selected applications in the two- and few-nucleon sectors. We highlight key challenges in developing high-precision three-body forces, such as the need to maintain consistency between two- and many-body interactions and constraints placed by the chiral and gauge symmetries after regularization.
Synchronization of Kuramoto oscillators in dense networks

We study synchronization properties of systems of Kuramoto oscillators. The problem can also be understood as a question about the properties of an energy landscape created by a graph. More formally, let G = (V, E) be a connected graph and (aij)i,j=1n denotes its adjacency matrix. Let the function f:Tn→R be given byf(θ1,…,θn)=∑i,j=1naijcos(θi−θj). This function has a global maximum when θi = θ for all 1 ⩽ i ⩽ n. It is known that if every vertex is connected to at least μ(n − 1) other vertices for μ sufficiently large, then every local maximum is global. Taylor proved this for μ ⩾ 0.9395 and Ling, Xu & Bandeira improved this to μ ⩾ 0.7929. We give a slight improvement to μ ⩾ 0.7889. Townsend, Stillman & Strogatz suggested that the critical value might be μc = 0.75.
Taking a Stance on Fake News: Towards Automatic Disinformation Assessment via Deep Bidirectional Transformer Language Models for Stance Detection

The exponential rise of social media and digital news in the past decade has had the unfortunate consequence of escalating what the United Nations has called a global topic of concern: the growing prevalence of disinformation. Given the complexity and time-consuming nature of combating disinformation through human assessment, one is motivated to explore harnessing AI solutions to automatically assess news articles for the presence of disinformation. A valuable first step towards automatic identification of disinformation is stance detection, where given a claim and a news article, the aim is to predict if the article agrees, disagrees, takes no position, or is unrelated to the claim. Existing approaches in literature have largely relied on hand-engineered features or shallow learned representations (e.g., word embeddings) to encode the claim-article pairs, which can limit the level of representational expressiveness needed to tackle the high complexity of disinformation identification. In this work, we explore the notion of harnessing large-scale deep bidirectional transformer language models for encoding claim-article pairs in an effort to construct state-of-the-art stance detection geared for identifying disinformation. Taking advantage of bidirectional cross-attention between claim-article pairs via pair encoding with self-attention, we construct a large-scale language model for stance detection by performing transfer learning on a RoBERTa deep bidirectional transformer language model, and were able to achieve state-of-the-art performance (weighted accuracy of 90.01%) on the Fake News Challenge Stage 1 (FNC-I) benchmark. These promising results serve as motivation for harnessing such large-scale language models as powerful building blocks for creating effective AI solutions to combat disinformation.
Data-Driven Recursive Least Squares Estimation for Model Predictive Current Control of Permanent Magnet Synchronous Motors

The performance of model predictive controllers (MPC) strongly depends on the quality of their models. In the field of electric drive control, white-box (WB) modeling approaches derived from first-order physical principles are most common. This procedure typically does not cover parasitic effects and often comes with parameter deviations. These issues are particularly crucial in the domain of self-commissioning drives where a hand-tailored, accurate WB plant model is not available. In order to compensate for such modeling errors and, consequently, to improve the control performance during transients and steady state, this article proposes a data-driven, real-time capable recursive least squares estimation method for the current control of a permanent magnet synchronous motor. Following this machine learning approach, the effect of the flux linkage and voltage harmonics due to the winding scheme can also be taken into account through suitable feature engineering. Moreover, a compensating scheme for the interlocking time of the inverter is proposed. The resulting algorithm is investigated using the well-known finite-control-set MPC (FCS-MPC) in the rotor-oriented coordinate system. The extensive experimental results show the superior performance of the presented scheme compared to a FCS-MPC-based on a state-of-the-art WB motor model using look-up tables for adressing (cross-)saturation.
Leveraging Self-supervised Denoising for Image Segmentation

Deep learning (DL) has arguably emerged as the method of choice for the detection and segmentation of biological structures in microscopy images. However, DL typically needs copious amounts of annotated training data that is for biomedical projects typically not available and excessively expensive to generate. Additionally, tasks become harder in the presence of noise, requiring even more high-quality training data. Hence, we propose to use denoising networks to improve the performance of other DL-based image segmentation methods. More specifically, we present ideas on how state-of-the-art self-supervised CARE networks can improve cell/nuclei segmentation in microscopy data. Using two state-of-the-art baseline methods, U-Net and StarDist, we show that our ideas consistently improve the quality of resulting segmentations, especially when only limited training data for noisy micrographs are available.
Vectorizing Quantum Turbulence Vortex-Core Lines for Real-Time Visualization

Vectorizing vortex-core lines is crucial for high-quality visualization and analysis of turbulence. While several techniques exist in the literature, they can only be applied to classical fluids. As quantum fluids with turbulence are gaining attention in physics, extracting and visualizing vortex-core lines for quantum fluids is increasingly desirable. In this article, we develop an efficient vortex-core line vectorization method for quantum fluids enabling real-time visualization of high-resolution quantum turbulence structure. From a dataset obtained through simulation, our technique first identifies vortex nodes based on the circulation field. To vectorize the vortex-core lines interpolating these vortex nodes, we propose a novel graph-based data structure, with iterative graph reduction and density-guided local optimization, to locate sub-grid-scale vortex-core line samples more precisely, which are then vectorized by continuous curves. This vortex-core representation naturally captures complex topology, such as branching during reconnection. Our vectorization approach reduces memory consumption by orders of magnitude, enabling real-time visualization performance. Different types of interactive visualizations are demonstrated to show the effectiveness of our technique, which could help further research on quantum turbulence.
Robustly Clustering a Mixture of Gaussians

We give an efficient algorithm for robustly clustering of a mixture of two arbitrary Gaussians, a central open problem in the theory of computationally efficient robust estimation, assuming only that the the means of the component Gaussians are well-separated or their covariances are well-separated. Our algorithm and analysis extend naturally to robustly clustering mixtures of well-separated strongly logconcave distributions. The mean separation required is close to the smallest possible to guarantee that most of the measure of each component can be separated by some hyperplane (for covariances, it is the same condition in the second degree polynomial kernel). We also show that for Gaussian mixtures, separation in total variation distance suffices to achieve robust clustering. Our main tools are a new identifiability criterion based on isotropic position and the Fisher discriminant, and a corresponding Sum-of-Squares convex programming relaxation, of fixed degree.
Palivizumab prophylaxis of RSV infections in Bosnia and Herzegovina

Background: Palivizumab is indicated for respiratory syncytial virus (RSV) prophylaxis in high-risk children. Methods: Observational study, based on 4 sites in Bosnia and Herzegovina (BH 365 (61.9%) infants in total were born before 33 weeks. Average gestational age of preterm infants enrolled for prematurity only was 30.2 ± 3.2 weeks; for preterm infants with BPD/CLD it was 28.3 ± 3.7 weeks. Overall average of palivizumab injections was 4.1 ± 1.0. Hospitalization rate related to severe lower respiratory infections (LRI) during the period of protection by palivizumab was 1.2%. Respiratory infections which deserved medical attention were observed in 3.7% infants included in palivizumab prophylaxis.Conclusion: RSV prophylaxis in B&H is provided systematically and successfully, following the national guidance established in 2009, with the aim of achieving a good cost-benefit ratio, with very low hospitalization rate for severe LRI in prophylaxed infants. New randomized controlled trials (RCTs) and American Academy of Pediatrics (AAP) guidance revised in 2014 will be taken into account in establishing a new national recommendation.
Improving Fictitious Play Reinforcement Learning with Expanding Models

Fictitious play with reinforcement learning is a general and effective framework for zero-sum games. However, using the current deep neural network models, the implementation of fictitious play faces crucial challenges. Neural network model training employs gradient descent approaches to update all connection weights, and thus is easy to forget the old opponents after training to beat the new opponents. Existing approaches often maintain a pool of historical policy models to avoid the forgetting. However, learning to beat a pool in stochastic games, i.e., a wide distribution over policy models, is either sample-consuming or insufficient to exploit all models with limited amount of samples. In this paper, we propose a learning process with neural fictitious play to alleviate the above issues. We train a single model as our policy model, which consists of sub-models and a selector. Everytime facing a new opponent, the model is expanded by adding a new sub-model, where only the new sub-model is updated instead of the whole model. At the same time, the selector is also updated to mix up the new sub-model with the previous ones at the state-level, so that the model is maintained as a behavior strategy instead of a wide distribution over policy models. Experiments on Kuhn poker, a grid-world Treasure Hunting game, and Mini-RTS environments show that the proposed approach alleviates the forgetting problem, and consequently improves the learning efficiency and the robustness of neural fictitious play.
Swimming at low Reynolds number

We address the swimming problem at low Reynolds number. This regime, which is typically used for micro-swimmers, is described by Stokes equations. We couple a PDE solver of Stokes equations, derived from the Feel++ ﬁnite elements library, to a quaternion-based rigid-body solver. We validate our numerical results both on a 2D exact solution and on an exact solution for a rotating rigid body respectively. Finally, we apply them to simulate the motion of a one-hinged swimmer, which obeys to the scallop theorem.
On the Stability of Lagrange Relative Equilibrium in The Planar Three-body Problem

Based on a novel coordinate system, we explore the stability of Lagrange relative equilibrium. First we give a system of practical equations of motion which is suitable for investigating the stability of relative equilibrium solutions. After that we prove that every relative equilibrium solution is always not stable in the sense of Lyapunov, we reduce the orbital stability of relative equilibrium solution to the Lyapunov stability of an equilibrium point. By the celebrated KAM theorem, we prove that Lagrange relative equilibrium is stable in the sense of measure, provided it's spectrally stable, except six special resonant cases. Indeed, under this condition, there are a great quantity of KAM invariant tori or quasi-periodic solutions in a small neighbourhood of Lagrange relative equilibrium. Furthermore, these tori or quasi-periodic solutions form a set whose relative measure rapidly tends to 1. We also investigate the effective (exponential) stability of Lagrange relative equilibrium by the celebrated Nekhoroshev's theory. We prove that Lagrange relative equilibrium is effectively stable for almost every choice of positive masses in the sense of measure, provided it's spectrally stable; and topologically, this is also right for a large open subset of spectrally stable space of masses.
THE JET/COUNTERJET SYMMETRY OF THE HH 212 OUTFLOW

We present Spitzer (IRAC) images and a VLT 2.1 μm image of the HH 212 outflow. We find that this outflow has a strong symmetry, with jet/counterjet knot pairs with ∆x < 1 00 position offsets. We deduce that the jet/counterjet knots are ejected with time differences ∆τ 0 ≈ 6 yr and velocity differences ∆v 0 ≈ 2 km s^−1 . We also analyze the deviations of the knot positions perpendicular to the outflow axis, and interpret them in terms of a binary orbital motion of the outflow source. Through this model, we deduce a ≈ 0.7 M mass for the outflow source, and a sep- aration of ≈ 80 AU between the components of the binary (assuming equal masses for the two components). Finally, using the IRAC data and the VLT 2.1 μm image we measure the proper motion velocities, obtaining values from 50 to 170 km s^−1 .
An Adaptive View of Adversarial Robustness from Test-time Smoothing Defense

The safety and robustness of learning-based decision-making systems are under threats from adversarial examples, as imperceptible perturbations can mislead neural networks to completely different outputs. In this paper, we present an adaptive view of the issue via evaluating various test-time smoothing defense against white-box untargeted adversarial examples. Through controlled experiments with pretrained ResNet-152 on ImageNet, we first illustrate the non-monotonic relation between adversarial attacks and smoothing defenses. Then at the dataset level, we observe large variance among samples and show that it is easy to inflate accuracy (even to 100%) or build large-scale (i.e., with size ~10^4) subsets on which a designated method outperforms others by a large margin. Finally at the sample level, as different adversarial examples require different degrees of defense, the potential advantages of iterative methods are also discussed. We hope this paper reveal useful behaviors of test-time defenses, which could help improve the evaluation process for adversarial robustness in the future.
Non-Autoregressive Video Captioning with Iterative Refinement

Existing state-of-the-art autoregressive video captioning methods (ARVC) generate captions sequentially, which leads to low inference efficiency. Moreover, the word-by-word generation process does not fit human intuition of comprehending video contents (i.e., first capturing the salient visual information and then generating well-organized descriptions), resulting in unsatisfied caption diversity. In order to press close to the human manner of comprehending video contents and writing captions, this paper proposes a non-autoregressive video captioning (NAVC) model with iterative refinement. We then further propose to exploit external auxiliary scoring information to assist the iterative refinement process, which can help the model focus on the inappropriate words more accurately. Experimental results on two mainstream benchmarks, i.e., MSVD and MSR-VTT, show that our proposed method generates more felicitous and diverse captions with a generally faster decoding speed, at the cost of up to 5% caption quality compared with the autoregressive counterpart. In particular, the proposal of using auxiliary scoring information not only improves non-autoregressive performance by a large margin, but is also beneficial for the caption diversity.
Data-Driven Wide-Area Control

We employ a novel data-enabled predictive control (DeePC) in voltage source converter (VSC) based high-voltage DC (HVDC) stations to perform safe and optimal wide-area control. Conventional optimal wide-area control is model-based. However, in practice detailed and accurate parametric power system models are rarely available. In contrast, the DeePC algorithm uses only input/output data measured from the unknown system to predict the future trajectories and calculate the optimal control policy. We showcase that the DeePC algorithm can effectively attenuate inter-area oscillations even in the presence of measurement noise, communication delays, nonlinear loads and uncertain load fluctuations. Furthermore, we derive a novel Min-Max DeePC algorithm to be applied independently in multiple VSC-HVDC stations to mitigate inter-area oscillations, which enables decentralized and robust optimal wide-area control. Further, we discuss how to relieve the computational burden of the Min-Max DeePC by reducing the dimension of prediction uncertainty. All of our results are illustrated with high-fidelity, nonlinear, and noisy simulations of a four-area test system.
2019 4 Reinforcement Learning Algorithm

Portfolio optimization is the process of managing a collection of assets to achieve optimal return with low risk in some consecutive trading periods. Although there are previous works on automating this process with reinforcement learning, the actions are limited to buy, hold and sell the assets. In this paper, we allow the agents to short the assets and our model accommodates buy/sell transaction costs. We implement two state-of-art reinforcement learning algorithms, Policy Gradient Actor Critic (PGAC) and Evolution Strategy (ES), both are widely-used and successful in game playing and robot control. We compare both algorithms and show that they out perform the benchmark.