Miracle-dz/UniFAR-FTUs huggingface dataset:
Characterizing Private Clipped Gradient Descent on Convex Generalized Linear Problems

Differentially private gradient descent (DP-GD) has been extremely effective both theoretically, and in practice, for solving private empirical risk minimization (ERM) problems. In this paper, we focus on understanding the impact of the clipping norm, a critical component of DP-GD, on its convergence. We provide the first formal convergence analysis of clipped DP-GD. More generally, we show that the value which one sets for clipping really matters: done wrong, it can dramatically affect the resulting quality; done properly, it can eliminate the dependence of convergence on the model dimensionality. We do this by showing a dichotomous behavior of the clipping norm. First, we show that if the clipping norm is set smaller than the optimal, even by a constant factor, the excess empirical risk for convex ERMs can increase from $O(1/n)$ to $\Omega(1)$, where $n$ is the number of data samples. Next, we show that, regardless of the value of the clipping norm, clipped DP-GD minimizes a well-defined convex objective over an unconstrained space, as long as the underlying ERM is a generalized linear problem. Furthermore, if the clipping norm is set within at most a constant factor higher than the optimal, then one can obtain an excess empirical risk guarantee that is independent of the dimensionality of the model space. Finally, we extend our result to non-convex generalized linear problems by showing that DP-GD reaches a first-order stationary point as long as the loss is smooth, and the convergence is independent of the dimensionality of the model space.
Data-driven Simulation and Optimization for Covid-19 Exit Strategies

The rapid spread of the Coronavirus SARS-2 is a major challenge that led almost all governments worldwide to take drastic measures to respond to the tragedy. Chief among those measures is the massive lockdown of entire countries and cities, which beyond its global economic impact has created some deep social and psychological tensions within populations. While the adopted mitigation measures (including the lockdown) have generally proven useful, policymakers are now facing a critical question: how and when to lift the mitigation measures? A carefully-planned exit strategy is indeed necessary to recover from the pandemic without risking a new outbreak. Classically, exit strategies rely on mathematical modeling to predict the effect of public health interventions. Such models are unfortunately known to be sensitive to some key parameters, which are usually set based on rules-of-thumb. In this paper, we propose to augment epidemiological forecasting with actual data-driven models that will learn to fine-tune predictions for different contexts (e.g., per country). We have therefore built a pandemic simulation and forecasting toolkit that combines a deep learning estimation of the epidemiological parameters of the disease in order to predict the cases and deaths, and a genetic algorithm component searching for optimal trade-offs/policies between constraints and objectives set by decision-makers. Replaying pandemic evolution in various countries, we experimentally show that our approach yields predictions with much lower error rates than pure epidemiological models in 75% of the cases and achieves a 95% R² score when the learning is transferred and tested on unseen countries. When used for forecasting, this approach provides actionable insights into the impact of individual measures and strategies.
Analysis of Flavonoids in Flower Petals of Soybean Flower Color Variants

We previously identified two anthocyanins, malvidin 3,5-di-O-glucoside and delphinidin 3,5-di-O-glucoside, in flower petals of soybean cultivars with purple flowers, 'Harosoy' and ''Clark'. However, two other anthocyanins found by high performance liquid chromatography (HPLC) analysis could not be identified. We further evaluated the effects of flower color genes W1, W3, W4, and Wm on flavonoid components using near-isogenic lines for these genes. The first objective of this study was to determine the two unidentified anthocyanins in flower petals of Clark. The second objective was to evaluate the effects of Wp for pink flower and W2 for purple-blue flower on flavonoid biosynthesis using purple-blue flowered cultivars Nezumisaya, Yogetsu-1-blue, and w2-20, and a pink flowered line LD05-15019-pink. Chemical analysis revealed that the two remaining anthocyanins were petunidin 3,5-di-O-glucoside and delphinidin 3-O-glucoside. Composition and amount of flavonoids in purple-blue flowers were generally similar to those of Clark suggesting that anthocyanin structure or copigmentation with other flavonoids may not be responsible for the color. Pink flowers contained 72% of total anthocyanins, 9% of the total flavonol glycosides, and 28% of aromadendrin 3-O-glucoside relative to Clark. Further, pink flowers lacked kaempferol 3-O-glucoside. It is uncertain whether the reduction in flavonol glycosides in combination with lower anthocyanin levels might fully explain the low intensity of pigmentation in pink flowers.
Evidence for magnon-phonon coupling in the topological magnet 
Cu3TeO6

We perform thermodynamic and inelastic neutron scattering (INS) measurements to study the lattice dynamics (phonons) of a cubic collinear antiferromagnet ${\mathrm{Cu}}_{3}{\mathrm{TeO}}_{6}$ which hosts topological spin excitations (magnons). While the specific heat and thermal conductivity results show that the thermal transport is dominated by phonons, the deviation of the thermal conductivity from a pure phononic model indicates that there is a strong coupling between magnons and phonons. In the INS measurements, we find a mode in the excitation spectra at 4.5 K, which exhibits a slight downward dispersion around the Brillouin zone center. This mode disappears above the N\'eel temperature and thus cannot be a phonon. Furthermore, the dispersion is distinct from that of a magnon. Instead, it can be explained by the magnon-polaron mode, collective excitations resulting from the hybridization between magnons and phonons. We consider the suppression of the thermal conductivity and emergence of the magnon-polaron mode to be evidence for magnon-phonon coupling in ${\mathrm{Cu}}_{3}{\mathrm{TeO}}_{6}$.
Search for the sub-stellar lithium depletion boundary in the open star cluster Coma Berenices

Aims. We mainly aim to search for the lithium depletion boundary (LDB) among the sub-stellar population of the open star cluster Coma Berenices. Methods. Since the number of brown dwarf candidates in Coma Ber available in the literature is scarce, we carried out a search for additional candidates photometrically using colour–magnitude diagrams combining optical and infrared photometry from the latest public releases of the following large-scale surveys: the United Kingdom InfraRed Telescope Infrared Deep Sky Survey (UKIRT/UKIDSS), the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS), the Sloan Digital Sky Survey (SDSS), and AllWISE. We checked astrometric consistency with cluster membership using Gaia DR2. A search for Li in three new and five previously known brown dwarf candidate cluster members was performed via spectroscopic observations using the OSIRIS instrument at the 10.4 m Gran Telescopio de Canarias (GTC). Results. A couple dozen new photometric candidate brown dwarfs located inside the tidal radius of Coma Ber are reported, but none of these are significantly fainter and cooler than previously known members. No LiI resonance doublet at 6707.8 Å was detected in any of eight Coma Ber targets in the magnitude range J = 15–19 and G = 20–23 observed with the GTC. Spectral types and radial velocities were derived from the GTC spectra. These values confirm the cluster membership of four L2–L2.5 dwarfs, two of which are new in the literature. Conclusions. The large Li depletion factors found among the four bona fide sub-stellar members in Coma Ber implies that the LDB must be located at spectral type later than L2.5 in this cluster. Using the latest evolutionary models for brown dwarfs, a lower limit of 550 Myr on the cluster age is set. This constraint has been combined with other dating methods to obtain an updated age estimate of 780 ± 230 Myr for the Coma Ber open cluster. Identification of significantly cooler sub-stellar cluster members in Coma Ber awaits the advent of the Euclid wide survey, which should reach a depth of about J = 23; this superb sensitivity will make it possible to determine the precise location of the sub-stellar LDB in this cluster and to carry out a complete census of its sub-stellar population.
A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization

In this paper, we study the performance of a large family of SGD variants in the smooth nonconvex regime. To this end, we propose a generic and flexible assumption capable of accurate modeling of the second moment of the stochastic gradient. Our assumption is satisfied by a large number of specific variants of SGD in the literature, including SGD with arbitrary sampling, SGD with compressed gradients, and a wide variety of variance-reduced SGD methods such as SVRG and SAGA. We provide a single convergence analysis for all methods that satisfy the proposed unified assumption, thereby offering a unified understanding of SGD variants in the nonconvex regime instead of relying on dedicated analyses of each variant. Moreover, our unified analysis is accurate enough to recover or improve upon the best-known convergence results of several classical methods, and also gives new convergence results for many new methods which arise as special cases. In the more general distributed/federated nonconvex optimization setup, we propose two new general algorithmic frameworks differing in whether direct gradient compression (DC) or compression of gradient differences (DIANA) is used. We show that all methods captured by these two frameworks also satisfy our unified assumption. Thus, our unified convergence analysis also captures a large variety of distributed methods utilizing compressed communication. Finally, we also provide a unified analysis for obtaining faster linear convergence rates in this nonconvex regime under the PL condition.
Zeroth-order Deterministic Policy Gradient

Deterministic Policy Gradient (DPG) removes a level of randomness from standard randomized-action Policy Gradient (PG), and demonstrates substantial empirical success for tackling complex dynamic problems involving Markov decision processes. At the same time, though, DPG loses its ability to learn in a model-free (i.e., actor-only) fashion, frequently necessitating the use of critics in order to obtain consistent estimates of the associated policy-reward gradient. In this work, we introduce Zeroth-order Deterministic Policy Gradient (ZDPG), which approximates policy-reward gradients via two-point stochastic evaluations of the $Q$-function, constructed by properly designed low-dimensional action-space perturbations. Exploiting the idea of random horizon rollouts for obtaining unbiased estimates of the $Q$-function, ZDPG lifts the dependence on critics and restores true model-free policy learning, while enjoying built-in and provable algorithmic stability. Additionally, we present new finite sample complexity bounds for ZDPG, which improve upon existing results by up to two orders of magnitude. Our findings are supported by several numerical experiments, which showcase the effectiveness of ZDPG in a practical setting, and its advantages over both PG and Baseline PG.
Deconfined criticality and ghost Fermi surfaces at the onset of antiferromagnetism in a metal

We propose a general theoretical framework, using two layers of ancilla qubits, for a continuous transition between a Fermi liquid with a large Fermi surface, and a pseudogap metal with a small Fermi surface of electron-like quasiparticles. The pseudogap metal can be a magnetically ordered metal, or a fractionalized Fermi liquid (FL*) without magnetic order. A critical 'ghost' Fermi surface emerges (alongside the large electron Fermi surface) at the quantum critical point, with the ghost fermions carrying neither spin nor charge, but minimally coupled to $(U(1) \times U(1))/Z_2$ or $(SU(2) \times U(1))/Z_2$ gauge fields. Away from the critical point on the pseudogap side, the ghost Fermi surface absorbs part of the large electron Fermi surface, and leads to a jump in the Hall co-efficient. We also find an example of an "unnecessary quantum critical point" between a metal with spin density order, and a metal with local moment magnetic order. The ghost fermions contribute a $T \ln (1/T)$ specific heat at temperature $T$ at the critical point, and could also be detected in other thermal probes. We relate our results to the phases of correlated electron compounds.
Few-shot Neural Architecture Search

To improve the search efficiency for Neural Architecture Search (NAS), One-shot NAS proposes to train a single super-net to approximate the performance of proposal architectures during search via weight-sharing. While this greatly reduces the computation cost, due to approximation error, the performance prediction by a single super-net is less accurate than training each proposal architecture from scratch, leading to search inefficiency. In this work, we propose few-shot NAS that explores the choice of using multiple super-nets: each super-net is pre-trained to be in charge of a sub-region of the search space. This reduces the prediction error of each super-net. Moreover, training these super-nets can be done jointly via sequential fine-tuning. A natural choice of sub-region is to follow the splitting of search space in NAS. We empirically evaluate our approach on three different tasks in NAS-Bench-201. Extensive results have demonstrated that few-shot NAS, using only 5 super-nets, significantly improves performance of many search methods with slight increase of search time. The architectures found by DARTs and ENAS with few-shot models achieved 88.53% and 86.50% test accuracy on CIFAR-10 in NAS-Bench-201, significantly outperformed their one-shot counterparts (with 54.30% and 54.30% test accuracy). Moreover, on AUTOGAN and DARTS, few-shot NAS also outperforms previously state-of-the-art models.
Ansor : Generating High-Performance Tensor Programs for Deep Learning

High-performance tensor programs are crucial to guarantee efficient execution of deep learning models. However, obtaining performant tensor programs for different operators on various hardware platforms is notoriously difficult. Currently, deep learning systems rely on vendor-provided kernel libraries or various search strategies to get performant tensor programs. These approaches either require significant engineering efforts in developing platform-specific optimization code or fall short in finding high-performance programs due to restricted search space and ineffective exploration strategy. We present Ansor, a tensor program generation framework for deep learning applications. Compared with existing search strategies, Ansor explores much more optimization combinations by sampling programs from a hierarchical representation of the search space. Ansor then fine-tunes the sampled programs with evolutionary search and a learned cost model to identify the best programs. Ansor can find high-performance programs that are outside the search space of existing state-of-the-art approaches. Besides, Ansor utilizes a scheduler to simultaneously optimize multiple subgraphs in a set of deep neural networks. Our evaluation shows that Ansor improves the execution performance of deep neural networks on the Intel CPU, ARM CPU, and NVIDIA GPU by up to $3.8\times$, $2.6\times$, and $1.7 \times$, respectively.
QCD2019 Workshop Summary

The topical workshop {\it Strong QCD from Hadron Structure Experiments} took place at Jefferson Lab from Nov. 6-9, 2019. Impressive progress in relating hadron structure observables to the strong QCD mechanisms has been achieved from the {\it ab initio} QCD description of hadron structure in a diverse array of methods in order to expose emergent phenomena via quasi-particle formation. The wealth of experimental data and the advances in hadron structure theory make it possible to gain insight into strong interaction dynamics in the regime of large quark-gluon coupling (the strong QCD regime), which will address the most challenging problems of the Standard Model on the nature of the dominant part of hadron mass, quark-gluon confinement, and the emergence of the ground and excited state hadrons, as well as atomic nuclei, from QCD. This workshop aimed to develop plans and to facilitate the future synergistic efforts between experimentalists, phenomenologists, and theorists working on studies of hadron spectroscopy and structure with the goal to connect the properties of hadrons and atomic nuclei available from data to the strong QCD dynamics underlying their emergence from QCD. These results pave the way for a future breakthrough extension in the studies of QCD with an Electron-Ion Collider in the U.S.
Towards Deeper Graph Neural Networks with Differentiable Group Normalization

Graph neural networks (GNNs), which learn the representation of a node by aggregating its neighbors, have become an effective computational tool in downstream applications. Over-smoothing is one of the key issues which limit the performance of GNNs as the number of layers increases. It is because the stacked aggregators would make node representations converge to indistinguishable vectors. Several attempts have been made to tackle the issue by bringing linked node pairs close and unlinked pairs distinct. However, they often ignore the intrinsic community structures and would result in sub-optimal performance. The representations of nodes within the same community/class need be similar to facilitate the classification, while different classes are expected to be separated in embedding space. To bridge the gap, we introduce two over-smoothing metrics and a novel technique, i.e., differentiable group normalization (DGN). It normalizes nodes within the same group independently to increase their smoothness, and separates node distributions among different groups to significantly alleviate the over-smoothing issue. Experiments on real-world datasets demonstrate that DGN makes GNN models more robust to over-smoothing and achieves better performance with deeper GNNs.
A Study of the Minimum Safe Distance between Human Driven and Driverless Cars Using Safe Distance Model

When driving,it is vital to maintain the right following distance between the vehicles to avoid rear-end collisions. The minimum safe distance depends on many factors, however, in this study the safe distance between the human-driven vehicles and a fully autonomous vehicle at a sudden stop by an automatic emergency brake was studied based on the human driver ability to react in an accident, the vehicles' braking system performance, and the speed of vehicles. For this approach, a safe distance car-following model was proposed to describe the safe distance between vehicles on a single lane dry road under conditions where both vehicles keep moving at a constant speed, and a lead autonomous vehicle suddenly stops by automatic emergency braking at an imminent incident. The proposed model then finally was being tested using MATLAB simulation, and results showed that confirmed the effectiveness of this model and the influence of driving speed and inter-vehicle distance on the rear-end collision was also indicated as well compared with the two and three seconds rule of safe following distance. The three seconds safe distance following rules is safe to be applied for all speed limits; however, the two seconds can be used on speed limits up to 45 Km/hr. A noticeable increase in rear-end collision was observed according to the simulation results if a car follows a driverless vehicle with two seconds rule above 45 km/hr.
Radio Emission from Ultra-stripped Supernovae as Diagnostics for Properties of the Remnant Double Neutron Star Binaries

An ultra-stripped supernova (SN) is an explosion of a helium or C+O star whose outer envelope has been stripped away by a companion neutron star. A double neutron star (DNS) binary is believed to be left after the explosion, which will emit the gravitational wave later at the coalescence. Recent detections of a few candidates for the ultra-stripped SN have constrained the properties of the explosion and the progenitor, but little information is given as to whether the remnant DNS binary will merge within the cosmic age. A large fraction of the material stripped away from the helium star through the binary interaction is expected to escape from the system and form circumstellar material (CSM). The CSM should be traced by radio emission induced by the collision with the SN ejecta. Based on the stellar evolution models previously developed, we calculate the expected radio luminosities from ultra-stripped SNe. We find that high radio luminosity at its maximum can be an indicator of small separation of a DNS binary leading to its merger within the cosmic age. Our results can be used to optimize the strategy for the radio follow-up observations such as observational epochs and frequencies.
Local-Area-Learning Network: Meaningful Local Areas for Efficient Point Cloud Analysis

Research in point cloud analysis with deep neural networks has made rapid progress in recent years. The pioneering work PointNet offered a direct analysis of point clouds. However, due to its architecture PointNet is not able to capture local structures. To overcome this drawback, the same authors have developed PointNet++ by applying PointNet to local areas. The local areas are defined by center points and their neighbors. In PointNet++ and its further developments the center points are determined with a Farthest Point Sampling (FPS) algorithm. This has the disadvantage that the center points in general do not have meaningful local areas. In this paper, we introduce the neural Local-Area-Learning Network (LocAL-Net) which places emphasis on the selection and characterization of the local areas. Our approach learns critical points that we use as center points. In order to strengthen the recognition of local structures, the points are given additional metric properties depending on the local areas. Finally, we derive and combine two global feature vectors, one from the whole point cloud and one from all local areas. Experiments on the datasets ModelNet10/40 and ShapeNet show that LocAL-Net is competitive for part segmentation. For classification LocAL-Net outperforms the state-of-the-arts.
Learning Diverse Representations for Fast Adaptation to Distribution Shift

The i.i. d. assumption is a useful idealization that underpins many successful approaches to supervised machine learning. However, its violation can lead to models that learn to exploit spurious correlations in the training data, rendering them vulnerable to adversarial interventions, undermining their reliability, and limiting their practical application. To mitigate this problem, we present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We propose a notion of diversity based on minimizing the conditional total correlation of final layer representations across models given the label, which we approximate using a variational estimator and minimize using adversarial training. To demonstrate our framework's ability to facilitate rapid adaptation to distribution shift, we train a number of simple classifiers from scratch on the frozen outputs of our models using a small amount of data from the shifted distribution. Under this evaluation protocol, our framework significantly outperforms a baseline trained using the empirical risk minimization principle.