Miracle-dz/UniFAR-FTUs huggingface dataset:
SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality

We propose a new framework, inspired by random matrix theory, for analyzing the dynamics of stochastic gradient descent (SGD) when both number of samples and dimensions are large. This framework applies to any fixed stepsize and the finite sum setting. Using this new framework, we show that the dynamics of SGD on a least squares problem with random data become deterministic in the large sample and dimensional limit. Furthermore, the limiting dynamics are governed by a Volterra integral equation. This model predicts that SGD undergoes a phase transition at an explicitly given critical stepsize that ultimately affects its convergence rate, which we also verify experimentally. Finally, when input data is isotropic, we provide explicit expressions for the dynamics and average-case convergence rates (i.e., the complexity of an algorithm averaged over all possible inputs). These rates show significant improvement over the worst-case complexities.
Nonexistence of the box dimension for dynamically invariant sets

One of the key challenges in the dimension theory of smooth dynamical systems is in establishing whether or not the Hausdorff, lower and upper box dimensions coincide for invariant sets. For sets invariant under conformal dynamics, these three dimensions always coincide. On the other hand, considerable attention has been given to examples of sets invariant under non-conformal dynamics whose Hausdorff and box dimensions do not coincide. These constructions exploit the fact that the Hausdorff and box dimensions quantify size in fundamentally different ways, the former in terms of covers by sets of varying diameters and the latter in terms of covers by sets of fixed diameters. In this article we construct the first example of a dynamically invariant set with distinct lower and upper box dimensions. Heuristically, this describes that if size is quantified in terms of covers by sets of equal diameters, a dynamically invariant set can appear bigger when viewed at certain resolutions than at others.
Scalar and gravity quantum field theories with fractional operators

We study a class of perturbative scalar and gravitational quantum field theories where dynamics is characterized by Lorentz-invariant or Lorentz-breaking non-local operators of fractional order and the underlying spacetime has a varying spectral dimension. These theories are either ghost free or power-counting renormalizable but they cannot be both at the same time. However, some of them are one-loop unitary and finite, and possibly unitary and finite at all orders. One of the theories is unitary and infrared-finite and can serve as a ghost-free model with large-scale modifications of general relativity.
Extracting the Locus of Attention at a Cocktail Party from Single-Trial EEG using a Joint CNN-LSTM Model

—Human brain performs remarkably well in seg-regating a particular speaker from interfering speakers in a multi-speaker scenario. It has been recently shown that we can quantitatively evaluate the segregation capability by modelling the relationship between the speech signals present in an auditory scene and the cortical signals of the listener measured using electroencephalography (EEG). This has opened up avenues to integrate neuro-feedback into hearing aids whereby the device can infer user’s attention and enhance the attended speaker. Commonly used algorithms to infer the auditory attention are based on linear systems theory where the speech cues such as envelopes are mapped on to the EEG signals. Here, we present a joint convolutional neural network (CNN) - long short-term memory (LSTM) model to infer the auditory attention. Our joint CNN-LSTM model takes the EEG signals and the spectrogram of the multiple speakers as inputs and classiﬁes the attention to one of the speakers. We evaluated the reliability of our neural network using three different datasets comprising of 61 subjects where, each subject undertook a dual-speaker experiment. The three datasets analysed corresponded to speech stimuli presented in three different languages namely German, Danish and Dutch. Using the proposed joint CNN-LSTM model, we obtained a median decoding accuracy of 77.2% at a trial duration of three seconds. Furthermore, we evaluated the amount of sparsity that our model can tolerate by means of magnitude pruning and found that the model can tolerate up to 50% sparsity without substantial loss of decoding accuracy.
Functional optimal transport: map estimation and domain adaptation for functional data

We introduce a formulation of optimal transport problem for distributions on function spaces, where the stochastic map between functional domains can be partially represented in terms of an (infinite-dimensional) Hilbert-Schmidt operator mapping a Hilbert space of functions to another. For numerous machine learning tasks, data can be naturally viewed as samples drawn from spaces of functions, such as curves and surfaces, in high dimensions. Optimal transport for functional data analysis provides a useful framework of treatment for such domains. { Since probability measures in infinite dimensional spaces generally lack absolute continuity (that is, with respect to non-degenerate Gaussian measures), the Monge map in the standard optimal transport theory for finite dimensional spaces may not exist. Our approach to the optimal transport problem in infinite dimensions is by a suitable regularization technique -- we restrict the class of transport maps to be a Hilbert-Schmidt space of operators. } To this end, we develop an efficient algorithm for finding the stochastic transport map between functional domains and provide theoretical guarantees on the existence, uniqueness, and consistency of our estimate for the Hilbert-Schmidt operator. We validate our method on synthetic datasets and examine the functional properties of the transport map. Experiments on real-world datasets of robot arm trajectories further demonstrate the effectiveness of our method on applications in domain adaptation.
Enabling Binary Neural Network Training on the Edge

The ever-growing computational demands of increasingly complex machine learning models frequently necessitate the use of powerful cloud-based infrastructure for their training. Binary neural networks are known to be promising candidates for on-device inference due to their extreme compute and memory savings over higher-precision alternatives. However, their existing training methods require the concurrent storage of high-precision activations for all layers, generally making learning on memory-constrained devices infeasible. In this article, we demonstrate that the backward propagation operations needed for binary neural network training are strongly robust to quantization, thereby making on-the-edge learning with modern models a practical proposition. We introduce a low-cost binary neural network training strategy exhibiting sizable memory footprint reductions while inducing little to no accuracy loss vs Courbariaux & Bengio’s standard approach. These decreases are primarily enabled through the retention of activations exclusively in binary format. Against the latter algorithm, our drop-in replacement sees memory requirement reductions of 3–5×, while reaching similar test accuracy (± 2 pp) in comparable time, across a range of small-scale models trained to classify popular datasets. We also demonstrate from-scratch ImageNet training of binarized ResNet-18, achieving a 3.78× memory reduction. Our work is open-source, and includes the Raspberry Pi-targeted prototype we used to verify our modeled memory decreases and capture the associated energy drops. Such savings will allow for unnecessary cloud offloading to be avoided, reducing latency, increasing energy efficiency, and safeguarding end-user privacy.
Non-linear frequency warping using constant-Q transformation for speech emotion recognition

In this work, we explore the constant-Q transform (CQT) for speech emotion recognition (SER). The CQT-based time-frequency analysis provides variable spectro-temporal resolution with higher frequency resolution at lower frequencies. Since lower-frequency regions of speech signal contain more emotion-related information than higher-frequency regions, the increased low-frequency resolution of CQT makes it more promising for SER than standard short-time Fourier transform (STFT). We present a comparative analysis of short-term acoustic features based on STFT and CQT for SER with deep neural network (DNN) as a back-end classifier. We optimize different parameters for both features. The CQT-based features outperform the STFT-based spectral features for SER experiments. Further experiments with cross-corpora evaluation demonstrate that the CQT-based systems provide better generalization with out-of-domain training data.
19F MAS-NMR Study of Structural Fluorine in Some Natural and Synthetic 2:1 Layer Silicates

High-resolution solid-state, fluorine-19, magic-angle spinning-nuclear magnetic resonance spectroscopy (MAS-NMR) was used to study natural and synthetic fluorinated 2:1 layer silicates of known composition. This technique enabled us to determine directly the coordination of structural fluorine and it was found to be sensitive to both the chemical nature of the octahedral elements (Al, Mg, Li) and the type of octahedral sheet (di- or trioctahedral). The observed chemical shifts at −132, −152, −176 and −182 ppm (relative to CFC13) were assigned to different environments of fluorine. The results were then used to characterize synthetic 2:1 layer silicates with unknown octahedral composition.
Digestible lysine levels in low-protein diets supplemented with synthetic amino acids for nursery, growing, and finishing barrows.

To investigate the effect of lysine level in low-protein diets (LPD) an experiment was conducted with 36 barrows fed 174, 140 and 118 g/kg of crude protein (CP), and three digestible lysine levels. Low-protein diets were supplemented with crystalline amino acids (AA) to the same concentration as in the standard CP diet. Growth performance, carcass characteristics and plasma urea nitrogen concentration were evaluated. In nursery pigs, the addition of lysine to LPD quadratically increased backfat thickness (BF; P < 0.01), longissimus muscle area (LMA; P < 0.001), and lean meat percentage (LMP; P < 0.05). In the growing phase, reducing dietary protein decreased average daily gain (ADG; P < 0.09), final body weight (BW; P < 0.09), fat free lean gain (FFLG; P < 0.09), and LMA (P < 0.038); and increased BF (P < 0.05). Addition of lysine to LPD quadratically increased ADG (P < 0.01), average daily feed intake (ADFI; P < 0.05), final BW (P < 0.006), and FFLG (P < 0.002). In the finishing period, pigs fed LPD had lower ADFI (P < 0.03) and feed:gain ratio (FGR; P < 0.003). Addition of lysine to LPD linearly increased BF (P < 0.04), and quadratically increased ADFI (P < 0.02), FGR (P < 0.009), BF (P < 0.08), and LMA (P < 0.08). The plasma urea nitrogen concentration was reduced (P < 0.05) in all phases in pigs fed LPD. Lysine requirement for fattening barrows fed LPD may be higher than the recommended concentration for standard CP diets.
Infinite GMRES for parameterized linear systems

We consider linear parameter-dependent systems $A(\mu) x(\mu) = b$ for many different $\mu$, where $A$ is large and sparse, and depends nonlinearly on $\mu$. Solving such systems individually for each $\mu$ would require great computational effort. In this work we propose to compute a partial parameterization $\tilde{x} \approx x(\mu)$ where $\tilde{x}(\mu)$ is cheap to compute for many different $\mu$. Our methods are based on the observation that a companion linearization can be formed where the dependence on $\mu$ is only linear. In particular, we develop methods which combine the well-established Krylov subspace method for linear systems, GMRES, with algorithms for nonlinear eigenvalue problems (NEPs) to generate a basis for the Krylov subspace. Within this new approach, the basis matrix is constructed in three different ways. We show convergence factor bounds obtained similarly to those for the method GMRES for linear systems. More specifically, a bound is obtained based on the magnitude of the parameter $\mu$ and the spectrum of the linear companion matrix, which corresponds to the reciprocal solutions to the corresponding NEP. Numerical experiments illustrate the competitiveness of our methods for large-scale problems.
Soccer Event Detection Using Deep Learning

Event detection is an important step in extracting knowledge from the video. In this paper, we propose a deep learning approach to detect events in a soccer match emphasizing the distinction between images of red and yellow cards and the correct detection of the images of selected events from other images. This method includes the following three modules: i) the variational autoencoder (VAE) module to differentiate between soccer images and others image, ii) the image classification module to classify the images of events, and iii) the fine-grain image classification module to classify the images of red and yellow cards. Additionally, a new dataset was introduced for soccer images classification that is employed to train the networks mentioned in the paper. In the final section, 10 UEFA Champions League matches are used to evaluate the networks' performance and precision in detecting the events. The experiments demonstrate that the proposed method achieves better performance than state-of-the-art methods.
Lightweight 3-D Localization and Mapping for Solid-State LiDAR

The LIght Detection And Ranging (LiDAR) sensor has become one of the most important perceptual devices due to its important role in simultaneous localization and mapping (SLAM). Existing SLAM methods are mainly developed for mechanical LiDAR sensors, which are often adopted by large scale robots. Recently, the solid-state LiDAR is introduced and becomes popular since it provides a cost-effective and lightweight solution for small scale robots. Compared to mechanical LiDAR, solid-state LiDAR sensors have higher update frequency and angular resolution, but also have smaller field of view (FoV), which is very challenging for existing LiDAR SLAM algorithms. Therefore, it is necessary to have a more robust and computationally efficient SLAM method for this new sensing device. To this end, we propose a new SLAM framework for solid-state LiDAR sensors, which involves feature extraction, odometry estimation, and probability map building. The proposed method is evaluated on a warehouse robot and a hand-held device. In the experiments, we demonstrate both the accuracy and efficiency of our method using an Intel L515 solid-state LiDAR. The results show that our method is able to provide precise localization and high quality mapping. We made the source codes public at https://github.com/wh200720041/SSL_SLAM.
A Generalized Model of Flocking with Steering

We introduce and analyze a model for the dynamics of flocking and steering of a finite number of agents. In this model, each agent's acceleration consists of flocking and steering components. The flocking component is a generalization of many of the existing models and allows for the incorporation of many real world features such as acceleration bounds, partial masking effects and orientation bias. The steering component is also integral to capture real world phenomena. We provide rigorous sufficient conditions under which the agents flock and steer together. We also provide a formal singular perturbation study of the situation where flocking happens much faster than steering. We end our work by providing some numerical simulations to illustrate our theoretical results.
Exploring the Limits of Few-Shot Link Prediction in Knowledge Graphs

Real-world knowledge graphs are often characterized by low-frequency relations—a challenge that has prompted an increasing interest in few-shot link prediction methods. These methods perform link prediction for a set of new relations, unseen during training, given only a few example facts of each relation at test time. In this work, we perform a systematic study on a spectrum of models derived by generalizing the current state of the art for few-shot link prediction, with the goal of probing the limits of learning in this few-shot setting. We find that a simple, zero-shot baseline — which ignores any relation-specific information — achieves surprisingly strong performance. Moreover, experiments on carefully crafted synthetic datasets show that having only a few examples of a relation fundamentally limits models from using fine-grained structural information and only allows for exploiting the coarse-grained positional information of entities. Together, our findings challenge the implicit assumptions and inductive biases of prior work and highlight new directions for research in this area.
Federated Reconstruction: Partially Local Federated Learning

Personalization methods in federated learning aim to balance the beneﬁts of federated and local training for data availability, communication cost, and robustness to client heterogeneity. Approaches that require clients to communicate all model parameters can be undesirable due to privacy and communication constraints. Other approaches require always-available or stateful clients, impractical in large-scale cross-device settings. We introduce Federated Reconstruction, the ﬁrst model-agnostic framework for partially local federated learning suitable for training and inference at scale. We motivate the framework via a connection to model-agnostic meta learning, empirically demonstrate its performance over existing approaches for collaborative ﬁltering and next word prediction, and release an open-source library for evaluating approaches in this setting. We also describe the successful deployment of this approach at scale for federated collaborative ﬁltering in a mobile keyboard application.
Toward establishing an abundant 
B
 and 
Bs
 spectrum up to the second orbital excitations

Stimulated by the exciting progress in experiments, we carry out a combined analysis of the masses, and strong and radiative decay properties of the B and Bs-meson states up to the second orbital excitations. Based on our good descriptions of the mass and decay properties for the low-lying well-established states B1(5721), B∗ 2 (5747), Bs1(5830) and B ∗ s2 (5840), we give a quark model classification for the high mass resonances observed in recent years. It is found that (i) the BJ(5840) resonance may be explained as the low mass mixed state B(|S D〉L) via 2S 1-1D1 mixing, or the pure B(2S 1) state, or B(2S 0). (ii) The BJ(5970) resonance may be assigned as the 1D3 state in the B meson family, although it as a pure 2 S 1 state cannot be excluded. (iii) The narrow structure around 6064 MeV observed in the B+K− mass spectrum at LHCb may be mainly caused by the BsJ(6109) resonance decaying into B ∗+K−, and favors the assignment of the high mass 1D-wave mixed state Bs(1D ′ 2 ) with J = 2−, although it as the 1D3 state cannot be excluded. (iv) The relatively broader BsJ(6114) structure observed at LHCb may be explained with the mixed state Bs(|S D〉H) via 2S 1-1D1 mixing, or a pure 1D1 state. Most of the missing 1P-, 1D-, and 2S -wave Band Bs-meson states have a relatively narrow width, they are most likely to be observed in their dominant decay channels with a larger data sample at LHCb.
Multimessenger Binary Mergers Containing Neutron Stars: Gravitational Waves, Jets, and γ-Ray Bursts

Neutron stars (NSs) are extraordinary not only because they are the densest form of matter in the visible Universe but also because they can generate magnetic fields ten orders of magnitude larger than those currently constructed on earth. The combination of extreme gravity with the enormous electromagnetic (EM) fields gives rise to spectacular phenomena like those observed on August 2017 with the merger of a binary neutron star system, an event that generated a gravitational wave (GW) signal, a short γ-ray burst (sGRB), and a kilonova. This event serves as the highlight so far of the era of multimessenger astronomy. In this review, we present the current state of our theoretical understanding of compact binary mergers containing NSs as gleaned from the latest general relativistic magnetohydrodynamic simulations. Such mergers can lead to events like the one on August 2017, GW170817, and its EM counterparts, GRB 170817 and AT 2017gfo. In addition to exploring the GW emission from binary black hole-neutron star and neutron star-neutron star mergers, we also focus on their counterpart EM signals. In particular, we are interested in identifying the conditions under which a relativistic jet can be launched following these mergers. Such a jet is an essential feature of most sGRB models and provides the main conduit of energy from the central object to the outer radiation regions. Jet properties, including their lifetimes and Poynting luminosities, the effects of the initial magnetic field geometries and spins of the coalescing NSs, as well as their governing equation of state, are discussed. Lastly, we present our current understanding of how the Blandford-Znajek mechanism arises from merger remnants as the trigger for launching jets, if, when and how a horizon is necessary for this mechanism, and the possibility that it can turn on in magnetized neutron ergostars, which contain ergoregions, but no horizons.
LOG-TRANSFORM AND THE WEAK HARNACK INEQUALITY FOR KINETIC FOKKER-PLANCK EQUATIONS

Abstract This article deals with kinetic Fokker–Planck equations with essentially bounded coefficients. A weak Harnack inequality for nonnegative super-solutions is derived by considering their log-transform and adapting an argument due to S. N. Kružkov (1963). Such a result rests on a new weak Poincaré inequality sharing similarities with the one introduced by W. Wang and L. Zhang in a series of works about ultraparabolic equations (2009, 2011, 2017). This functional inequality is combined with a classical covering argument recently adapted by L. Silvestre and the second author (2020) to kinetic equations.
Fission fragment distributions and their impact on the 
r
-process nucleosynthesis in neutron star mergers

Neutron star (NS) merger ejecta offer viable sites for the production of heavy r -process elements with nuclear mass numbers A (cid:2) 140. The crucial role of ﬁssion recycling is responsible for the robustness of this site against many astrophysical uncertainties. Here, we introduce improvements to our scission-point model, called SPY, to derive the ﬁssion fragment distribution for all neutron-rich ﬁssioning nuclei of relevance in r -process calculations. These improvements include a phenomenological modiﬁcation of the scission distance and a smoothing procedure of the distribution. Such corrections lead to much better agreement with experimental ﬁssion yields. Those yields are also used to estimate the number of neutrons emitted by the excited fragments on the basis of different neutron evaporation models. Our ﬁssion yields are extensively compared to those predicted by the GEF (general description of ﬁssion observables) model. The impact of ﬁssion on the r -process nucleosynthesis in binary neutron mergers is also reanalyzed. Two scenarios are considered, the ﬁrst one with low initial electron fraction subject to intense ﬁssion recycling, in contrast to the second one, which includes weak interactions on nucleons. The various regions of the nuclear chart responsible for ﬁssion recycling during the neutron irradiation and after freeze-out are discussed. The contribution ﬁssion processes may have to the ﬁnal abundance distribution is also studied in detail in the light of newly deﬁned quantitative indicators describing the ﬁssion recycling, the ﬁssion seeds, and the ﬁssion progenitors. In particular, those allow us to estimate the contribution of ﬁssion to the ﬁnal abundance distribution stemming from speciﬁc heavy nuclei. Calculations obtained with SPY and GEF ﬁssion fragment distributions are compared for both r -process scenarios.