Miracle-dz/UniFAR-FTUs huggingface dataset:
Neural-Network Quantum State of Transverse-Field Ising Model

Along the way initiated by Carleo and Troyer [G. Carleo and M. Troyer, Science 355 (2017) 602], we construct the neural-network quantum state of transverse-field Ising model (TFIM) by an unsupervised machine learning method. Such a wave function is a map from the spin-configuration space to the complex number field determined by an array of network parameters. To get the ground state of the system, values of the network parameters are calculated by a Stochastic Reconfiguration (SR) method. We provide for this SR method an understanding from action principle and information geometry aspects. With this quantum state, we calculate key observables of the system, the energy, correlation function, correlation length, magnetic moment, and susceptibility. As innovations, we provide a high efficiency method and use it to calculate entanglement entropy (EE) of the system and get results consistent with previous work very well.
Magnetic Control of Flexible Thermoelectric Devices Based on Macroscopic 3D Interconnected Nanowire Networks

Spin‐related effects in thermoelectricity can be used to design more efficient refrigerators and offer promising applications for the harvesting of thermal energy. The key challenge is to design structural and compositional magnetic material systems with sufficiently high efficiency and power output for transforming thermal energy into electric energy and vice versa. The fabrication of large‐area 3D interconnected Co/Cu nanowire networks is demonstrated, thereby enabling the controlled Peltier cooling of macroscopic electronic components with an external magnetic field. The flexible, macroscopic devices overcome the inherent limitations of nanoscale magnetic structures that are caused by insufficient power generation capability limiting the heat management applications. From properly designed experiments, large spin‐dependent Seebeck and Peltier coefficients of −9.4 µV K−1 and −2.8 mV at room temperature, respectively, are found. The resulting power factor of Co/Cu nanowire networks at room temperature (≈ 7.5 mW K−2 m−1) is larger than those of state‐of‐the‐art thermoelectric materials, such as BiTe alloys, and the magnetopower factor ratio reaches about 100% over a wide temperature range. Validation of magnetic control of heat flow achieved by taking advantage of the spin‐dependent thermoelectric properties of flexible macroscopic nanowire networks lays the groundwork to design shapeable thermoelectric coolers exploiting the spin degree of freedom.
Training neural networks to have brain-like representations improves object recognition performance

The current state-of-the-art object recognition algorithms, deep convolutional neural networks (DCNNs), are inspired by the architecture of the mammalian visual system [8], and capable of human-level performance on many tasks [15]. However, even these algorithms make errors. As DCNNs improve at object recognition tasks, they develop representations in their hidden layers that become more similar to those observed in the mammalian brains [24]. This led us to hypothesize that teaching DCNNs to achieve even more brain-like representations could improve their performance. To test this, we trained DCNNs on a composite task, wherein networks were trained to: a) classify images of objects; while b) having intermediate representations that resemble those observed in neural recordings from monkey visual cortex. Compared with DCNNs trained purely for object categorization, DCNNs trained on the composite task had better object recognition performance. Our results outline a new way to regularize object recognition networks, using transfer learning strategies in which the brain serves as a teacher for training DCNNs.
Jet-driven Galaxy-scale Gas Outflows in the Hyperluminous Quasar 3C 273

We present an unprecedented view of the morphology and kinematics of the extended narrow-line region (ENLR) and molecular gas around the prototypical hyperluminous quasar 3C 273 (Lbol ∼ 1047 erg s−1 at z = 0.158) based on VLT-MUSE optical 3D spectroscopy and ALMA observations. We find the following: (1) the ENLR size of 12.1 ± 0.2 kpc implies a smooth continuation of the size–luminosity relation out to large radii or a much larger break radius as previously proposed. (2) The kinematically disturbed ionized gas with line splits reaching 1000 km s−1 out to 6.1 ± 1.5 kpc is aligned along the jet axis. (3) The extreme line broadening on kiloparsec scales is caused by the spatial and spectral blending of many distinct gas clouds separated on subarcsecond scales by different line-of-sight (LOS) velocities. The ENLR velocity field combined with the known jet orientation rules out a simple scenario of a radiatively driven radial expansion of the outflow. Instead, we propose that a pressurized expanding hot gas cocoon created by the radio jet is impacting on an inclined gas disk, leading to transverse and/or backflow motion with respect to our LOS. The molecular gas morphology may be explained by either a density wave at the front of the outflow expanding along the jet direction as predicted by the positive feedback scenario or the cold gas may be trapped in a stellar overdensity caused by a recent merger event. Using 3C 273 as a template for observations of high-redshift hyperluminous quasars reveals that large-scale ENLRs and kiloparsec-scale outflows may often be missed, due to the brightness of the nuclei and the limited sensitivity of current near-IR instrumentation.
Quantifying Exposure Bias for Neural Language Generation

The exposure bias problem refers to the training-generation discrepancy, caused by teacher forcing, in maximum likelihood estimation (MLE) training for auto-regressive neural network language models (LM). It has been regarded as a central problem for neural language generation (NLG) model training. Although a lot of algorithms have been proposed to avoid teacher forcing and therefore` to alleviate exposure bias, there is little work showing how serious the exposure bias problem actually is. In this work, we first identify the self-recovery ability of MLE-trained LM, which casts doubt on the seriousness of exposure bias. We then propose sequence-level (EB-bleu) and word-level (EB-C) metrics as reasonable proxies to quantify the impact of exposure bias. We conduct experiments for the LSTM/transformer model, in both real and synthetic settings. In addition to the unconditional NLG task, we also include results for a seq2seq machine translation task. Surprisingly, all our measurements suggest that removing the training-generation discrepancy only brings very little performance gain. In our analysis, we hypothesise that although there exist a mismatch between the model distribution and the data distribution, the mismatch is still in the model's "comfortable zone", and is not big enough to induce significant performance loss.
Regularized Weighted Chebyshev Approximations for Support Estimation

We introduce a new method for estimating the support size of an unknown distribution which provably matches the performance bounds of the state-of-the-art techniques in the area and outperforms them in practice. In particular, we present both theoretical and computer simulation results that illustrate the utility and performance improvements of our method. The theoretical analysis relies on introducing a new weighted Chebyshev polynomial approximation method, jointly optimizing the bias and variance components of the risk, and combining the weighted minmax polynomial approximation method with discretized semi-infinite programming solvers. Such a setting allows for casting the estimation problem as a linear program (LP) with a small number of variables and constraints that may be solved as efficiently as the original Chebyshev approximation problem. Our technique is tested on synthetic data and used to address an important problem in computational biology - estimating the number of bacterial genera in the human gut. On synthetic datasets, for practically relevant sample sizes, we observe significant improvements in the value of the worst-case risk compared to existing methods. For the bioinformatics application, using metagenomic data from the NIH Human Gut and the American Gut Microbiome Projects, we generate a list of frequencies of bacterial taxa that allows us to estimate the number of bacterial genera to approximately 2300.
Ascorbylperoxide Contaminating Parenteral Nutrition Perturbs the Lipid Metabolism in Newborn Guinea Pig

The light exposure of parenteral nutritive solutions generates peroxides such as H2O2 and ascorbylperoxide [2,3-diketo-4-hydoxyperoxyl-5,6-dihydroxyhexanoic acid]. This absence of photoprotection is associated with higher plasma triacylglycerol (TG) concentration in premature infants and oxidative stress and H2O2-independent hepatic steatosis in animals. We hypothesized that ascorbylperoxide is the active agent leading to high TG. The aim was to investigate the role of ascorbylperoxide in glucose and lipid metabolism in an animal model of neonatal parenteral nutrition. Three-day-old guinea pigs received through a catheter in the jugular solutions containing dextrose plus 0, 90, 225, or 450 μM ascorbylperoxide. After 4 days, blood and liver were sampled and treated for determinations of TG, cholesterol, markers of oxidative stress (redox potential of glutathione and F2α-isoprostane), and activities and protein levels of acetyl-CoA carboxylase (ACC), glucokinase, and phosphofructokinase (PFK). Ascorbylperoxide concentration was measured in urine on the last day. Data were compared by analysis of variance (p < 0.05). Plasma TG and cholesterol and hepatic PFK activity increased (200% of control), whereas ACC activity decreased (66% of control) in the function of the amount of ascorbylperoxide infused. Both markers of oxidative stress were higher in animals receiving the highest amounts of ascorbylperoxide. The logarithmic relations between urinary ascorbylperoxide and plasma TG (r2 = 0.69) and hepatic PFK activity (r2 = 0.26) were positive, whereas they were negative with ACC activity (r2 = 0.50). In conclusion, ascorbylperoxide contaminating parenteral nutrition stimulates glycolysis, allowing higher availability of substrates for lipid synthesis. The logarithmic relation between urinary ascorbylperoxide and plasma TG suggests a very low efficient concentration.
Combating Label Noise in Deep Learning Using Abstention

We introduce a novel method to combat label noise when training deep neural networks for classification. We propose a loss function that permits abstention during training thereby allowing the DNN to abstain on confusing samples while continuing to learn and improve classification performance on the non-abstained samples. We show how such a deep abstaining classifier (DAC) can be used for robust learning in the presence of different types of label noise. In the case of structured or systematic label noise -- where noisy training labels or confusing examples are correlated with underlying features of the data-- training with abstention enables representation learning for features that are associated with unreliable labels. In the case of unstructured (arbitrary) label noise, abstention during training enables the DAC to be used as an effective data cleaner by identifying samples that are likely to have label noise. We provide analytical results on the loss function behavior that enable dynamic adaption of abstention rates based on learning progress during training. We demonstrate the utility of the deep abstaining classifier for various image classification tasks under different types of label noise; in the case of arbitrary label noise, we show significant improvements over previously published results on multiple image benchmarks. Source code is available at this https URL
Carbon-rich (DQ) white dwarfs in the Sloan Digital Sky Survey

Context.Among the spectroscopically identified white dwarfs, a fraction smaller than 2% have spectra dominated by carbon lines, mainly molecular C2, but also a smaller group dominated by C Iand C IIlines. These are together called DQ white dwarfs.Aims. We want to derive atmospheric parametersTeff, logg, and carbon abundances for a large sample of these stars and discuss implications for their spectral evolution.Methods. Sloan Digital Sky Survey spectra andugrizphotometry were used, together withGaiaData Release 2 parallaxes andGband photometry. These were fitted to synthetic spectra and theoretical photometry derived from model atmospheres.Results. We found that the DQ hotter thanTeff~ 10 000 K have masses ~ 0.4M⊙larger than the classical DQ, which have masses typical for the majority of white dwarfs (~ 0.6M⊙). We found some evidence that the peculiar DQ below 10 000 K also have significantly larger masses and may thus be the descendants of the hot and warm DQ above 10 000 K. A significant fraction of the hotter objects withTeff> 14 500 K have atmospheres dominated by carbon.
The Fourth Catalog of Active Galactic Nuclei Detected by the Fermi Large Area Telescope

The fourth catalog of active galactic nuclei (AGNs) detected by the Fermi Gamma-ray Space Telescope Large Area Telescope (4LAC) between 2008 August 4 and 2016 August 2 contains objects located at high Galactic latitudes ( ). It includes 85% more sources than the previous 3LAC catalog based on 4 yr of data. AGNs represent at least 79% of the high-latitude sources in the fourth Fermi-Large Area Telescope Source Catalog (4FGL), which covers the energy range from 50 MeV to 1 TeV. In addition, gamma-ray AGNs are found at low Galactic latitudes. Most of the 4LAC AGNs are blazars (98%), while the remainder are other types of AGNs. The blazar population consists of 24% Flat Spectrum Radio Quasars (FSRQs), 38% BL Lac-type objects, and 38% blazar candidates of unknown types (BCUs). On average, FSRQs display softer spectra and stronger variability in the gamma-ray band than BL Lacs do, confirming previous findings. All AGNs detected by ground-based atmospheric Cerenkov telescopes are also found in the 4LAC.
Are Sixteen Heads Really Better than One?

Attention is a powerful and ubiquitous mechanism for allowing neural models to focus on particular salient pieces of information by taking their weighted average when making predictions. In particular, multi-headed attention is a driving force behind many recent state-of-the-art NLP models such as Transformer-based MT models and BERT. These models apply multiple attention mechanisms in parallel, with each attention "head" potentially focusing on different parts of the input, which makes it possible to express sophisticated functions beyond the simple weighted average. In this paper we make the surprising observation that even if models have been trained using multiple heads, in practice, a large percentage of attention heads can be removed at test time without significantly impacting performance. In fact, some layers can even be reduced to a single head. We further examine greedy algorithms for pruning down models, and the potential speed, memory efficiency, and accuracy improvements obtainable therefrom. Finally, we analyze the results with respect to which parts of the model are more reliant on having multiple heads, and provide precursory evidence that training dynamics play a role in the gains provided by multi-head attention.
Cold Case: The Lost MNIST Digits

Although the popular MNIST dataset [LeCun et al., 1994] is derived from the NIST database [Grother and Hanaoka, 1995], the precise processing steps for this derivation have been lost to time. We propose a reconstruction that is accurate enough to serve as a replacement for the MNIST dataset, with insignificant changes in accuracy. We trace each MNIST digit to its NIST source and its rich metadata such as writer identifier, partition identifier, etc. We also reconstruct the complete MNIST test set with 60,000 samples instead of the usual 10,000. Since the balance 50,000 were never distributed, they enable us to investigate the impact of twenty-five years of MNIST experiments on the reported testing performances. Our results unambiguously confirm the trends observed by Recht et al. [2018, 2019]: although the misclassification rates are slightly off, classifier ordering and model selection remain broadly reliable. We attribute this phenomenon to the pairing benefits of comparing classifiers on the same digits.
Fair Resource Allocation in Federated Learning

Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency.
Toward Self-Supervised Object Detection in Unlabeled Videos

Unlabeled video in the wild presents a valuable, yet so far unharnessed, source of information for learning vision tasks. We present the first attempt of fully self-supervised learning of object detection from subtitled videos without any manual object annotation. To this end, we use the How2 multi-modal collection of instructional videos with English subtitles. We pose the problem as learning with a weakly- and noisily-labeled data, and propose a novel training model that can confront high noise levels, and yet train a classifier to localize the object of interest in the video frames, without any manual labeling involved. We evaluate our approach on a set of 11 manually annotated objects in over 5000 frames and compare it to an existing weakly-supervised approach as baseline. Benchmark data and code will be released upon acceptance of the paper.
A neuromorphic boost to RNNs using low pass filters

The increasing difficulty with Moore's law scaling and the remarkable success of machine learning have triggered a renaissance in the study of low-latency, energy-efficient accelerators for machine learning applications. In particular, spiking neural networks (SNNs) and their neuromorphic hardware implementations have started to receive substantial attention from academia and industry. However, SNNs perform relatively poorly compared to their rate-based counterparts, in terms of accuracy in pattern recognition tasks. In this paper, we show that the addition of a low pass filtering term to a recurrent neural network enables mapping to neuromorphic SNN devices. This breakthrough will allow the implementation of much more sophisticated SNN models on neuromorphic hardware than the state of the art. The use of low-power neuromorphic platforms will, in turn, enable the construction of compact devices that can perform always-on processing in ultra-low power edge computing applications. We further show that the low pass filtered RNN cell matches and outperforms their unfiltered variants in a range of tasks. Finally, we argue that the low pass filter is a temporal regularizer, allowing RNNs to learn and generalise better than their unfiltered variants.
Effects of Online Self-Regulation Activities on Physical Activity Among Pregnant and Early Postpartum Women

Physical and psychological changes that occur during pregnancy present a unique challenge for women's physical activity. Using a theory-based prospective design, this study examines the effects of pregnant women's (a) physical activity cognitions (self-efficacy, outcome expectancy, and safety beliefs) and (b) online self-regulation activities (goal-setting and self-monitoring) on subsequent changes in their physical activity intentions and behavior during pregnancy and immediately postpartum. The authors used data from three panel surveys administered to pregnant women enrolled in a web-based intervention to promote healthy pregnancy and postpartum weight, as well as log data on their use of self-regulatory features on the intervention website. Perceived self-efficacy and perceived safety of physical activity in pregnancy enhanced subsequent intentions to be physically active. Repeated goal-setting and monitoring of those goals helped to maintain positive intentions during pregnancy, but only repeated self-monitoring transferred positive intentions into actual behavior. Theoretically, this study offers a better understanding of the roles of self-regulation activities in the processes of goal-striving. The authors also discuss practical implications for encouraging physical activity among pregnant and early postpartum women.
Understanding Generalization of Deep Neural Networks Trained with Noisy Labels

Over-parameterized deep neural networks trained by simple first-order methods are known to be able to fit any labeling of data. When the training dataset contains a fraction of noisy labels, can neural networks be resistant to over-fitting and still generalize on the true distribution? Inspired by recent theoretical work that established connections between over-parameterized neural networks and neural tangent kernel (NTK), we propose two simple regularization methods for this purpose: (i) regularization by the distance between the network parameters to initialization, and (ii) adding a trainable auxiliary variable to the network output for each training example. Theoretically, both methods are related to kernel ridge regression with respect to the NTK, and we prove their generalization guarantee on the true data distribution despite being trained using noisy labels. The generalization bound is independent of the network size, and only depends on the training inputs and true labels (instead of noisy labels) as well as the noise level in the labels. Empirical results verify the effectiveness of these methods on noisily labeled datasets.
Uniform a priori estimates for positive solutions of higher order Lane-Emden equations in $\mathbb{R}^n$

In this paper, we study the existence of uniform a priori estimates for positive solutions to Navier problems of higher order Lane-Emden equations \begin{equation*} (-\Delta)^{m}u(x)=u^{p}(x), \qquad \,\, x\in\Omega \end{equation*} for all large exponents $p$, where $\Omega\subset\mathbb{R}^{n}$ is a star-shaped or strictly convex bounded domain with $C^{2m-2}$ boundary, $n\geq4$ and $2\leq m\leq\frac{n}{2}$. Our results extend those of previous authors for second order $m=1$ to general higher order cases $m\geq2$.
Dissection of the protein G B1 domain binding site for human IgG Fc fragment

The contribution to the free energy of binding of each of the residues forming the binding site for a human IgG Fc fragment on the surface of the B1 domain of protein G was determined by alanine‐scanning mutagenesis. The interface between these two proteins is atypical in that it is smaller than usual, polar in character, and involves two well‐defined “knobs‐into‐holes” interactions. The bulk of the free energy of binding is contributed by three central residues, which make hydrogen bonds across the interface. Of these, the most critical interaction is formed by Glu27, which acts as a charged knob on the surface of the B1 domain, inserting into a polar hole on the Fc fragment. A single alanine mutation of this residue virtually abolishes stable complex formation. Formation of a stable interface between these two proteins is therefore dominated by a small, polar “hot spot. ”