Miracle-dz/UniFAR-FTUs huggingface dataset:
Sparse Implicit Processes for Approximate Inference

Implicit Processes (IPs) are ﬂexible priors that can describe models such as Bayesian neural networks, neural samplers and data generators. IPs allow for approximate inference in function-space. This avoids some degenerate problems of parameter-space approximate inference due to the high number of parameters and strong dependencies. For this, an extra IP is often used to approximate the posterior of the prior IP. However, simultaneously adjusting the parameters of the prior IP and the approximate posterior IP is a challenging task. Existing methods that can tune the prior IP result in a Gaussian predictive distribution, which fails to capture important data patterns. By contrast, methods producing ﬂexible predictive distributions by using another IP to approximate the posterior process cannot ﬁt the prior IP to the observed data. We pro-pose here a method that can carry out both tasks. For this, we rely on an inducing-point representation of the prior IP, as often done in the context of sparse Gaussian processes. The result is a scalable method for approximate inference with IPs that can tune the prior IP parameters to the data, and that provides accurate non-Gaussian predictive distributions.
Overall spectral properties of prompt emissions with diverse segments in Swift/BAT short gamma-ray bursts

Owing to lack of multiple components of prompt $\gamma$-ray emissions in short gamma-ray bursts (sGRBs), how these distinct components are correlated still keeps unclear. In this paper, we investigate the spectral and temporal properties of precursors, main peaks and extended emissions in 26 sGRBs including GRB 170817A. It is found that peak energies ($E_p$) in each pulse are uncorrelated with the pulse duration ($t_{dur}$). Meanwhile, we find that there is no obvious correlation between peak energy and energy fluence. Interestingly, there is no obvious spectral evolution from earlier precursors to later extended emissions in view of the correlations of $t_{dur}$ with either the $E_p$ or the low energy spectrum index, $\alpha$. A power-law correlation between the average flux ($F_{p}$) and the energy fluence ($S_\gamma$), $log F_p=(0.62\pm0.07) log S_\gamma + (0.27\pm0.07)$, is found to exist in the individual segments instead of mean peaks previously. Furthermore, we also find that the main peaks are on average brighter than the precursors or the extend emissions about one order of magnitude. On the basis of all the above analyses, one can conclude that three emissive components would share the same radiation mechanisms but they might be dominated by diverse physical processes.
Alternative Input Signals Ease Transfer in Multilingual Machine Translation

Recent work in multilingual machine translation (MMT) has focused on the potential of positive transfer between languages, particularly cases where higher-resourced languages can benefit lower-resourced ones. While training an MMT model, the supervision signals learned from one language pair can be transferred to the other via the tokens shared by multiple source languages. However, the transfer is inhibited when the token overlap among source languages is small, which manifests naturally when languages use different writing systems. In this paper, we tackle inhibited transfer by augmenting the training data with alternative signals that unify different writing systems, such as phonetic, romanized, and transliterated input. We test these signals on Indic and Turkic languages, two language families where the writing systems differ but languages still share common features. Our results indicate that a straightforward multi-source self-ensemble – training a model on a mixture of various signals and ensembling the outputs of the same model fed with different signals during inference, outperforms strong ensemble baselines by 1.3 BLEU points on both language families. Further, we find that incorporating alternative inputs via self-ensemble can be particularly effective when training set is small, leading to +5 BLEU when only 5% of the total training data is accessible. Finally, our analysis demonstrates that including alternative signals yields more consistency and translates named entities more accurately, which is crucial for increased factuality of automated systems.
Graph Neural Networks with Learnable Structural and Positional Representations

Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce Positional Encoding (PE) of nodes, and inject it into the input layer, like in Transformers. Possible graph PE are Laplacian eigenvectors. In this work, we propose to decouple structural and positional representations to make easy for the network to learn these two essential properties. We introduce a novel generic architecture which we call LSPE (Learnable Structural and Positional Encodings). We investigate several sparse and fully-connected (Transformer-like) GNNs, and observe a performance increase for molecular datasets, from 1.79% up to 64.14% when considering learnable PE for both GNN classes.
Superconducting dome in ferroelectric-type materials from soft mode instability

We present a minimal theory of superconductivity enhancement in ferroelectric-type materials. Simple expressions for the optical mode responsible for the soft mode transition are assumed. A key role is played by the anharmonic phonon damping which is modulated by an external control parameter (electron doping or mechanical strain) causing the appearance of the soft mode. It is shown that the enhancement in the superconducting critical temperature $T_{c}$ upon approaching the ferroelectric transition from either side is due to the Stokes electron-phonon scattering processes promoted by strong phonon damping effects.
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes

When designing Convolutional Neural Networks (CNNs), one must select the size\break of the convolutional kernels before training. Recent works show CNNs benefit from different kernel sizes at different layers, but exploring all possible combinations is unfeasible in practice. A more efficient approach is to learn the kernel size during training. However, existing works that learn the kernel size have a limited bandwidth. These approaches scale kernels by dilation, and thus the detail they can describe is limited. In this work, we propose FlexConv, a novel convolutional operation with which high bandwidth convolutional kernels of learnable kernel size can be learned at a fixed parameter cost. FlexNets model long-term dependencies without the use of pooling, achieve state-of-the-art performance on several sequential datasets, outperform recent works with learned kernel sizes, and are competitive with much deeper ResNets on image benchmark datasets. Additionally, FlexNets can be deployed at higher resolutions than those seen during training. To avoid aliasing, we propose a novel kernel parameterization with which the frequency of the kernels can be analytically controlled. Our novel kernel parameterization shows higher descriptive power and faster convergence speed than existing parameterizations. This leads to important improvements in classification accuracy.
Attention-Free Keyword Spotting

Till now, attention-based models have been used with great success in the keyword spotting problem domain. However, in light of recent advances in deep learning, the question arises whether self-attention is truly irreplaceable for recognizing speech keywords. We thus explore the usage of gated MLPs—previously shown to be alternatives to transformers in vision tasks—for the keyword spotting task. We provide a family of highly efﬁcient MLP-based models for keyword spotting, with less than 0.5 million parameters. We show that our approach achieves competitive performance on Google Speech Commands V2-12 and V2-35 benchmarks with much fewer parameters than self-attention-based methods.
Equivariant and Invariant Reynolds Networks

Invariant and equivariant networks are useful in learning data with symmetry, including images, sets, point clouds, and graphs. In this paper, we consider invariant and equivariant networks for symmetries of finite groups. Invariant and equivariant networks have been constructed by various researchers using Reynolds operators. However, Reynolds operators are computationally expensive when the order of the group is large because they use the sum over the whole group, which poses an implementation difficulty. To overcome this difficulty, we consider representing the Reynolds operator as a sum over a subset instead of a sum over the whole group. We call such a subset a Reynolds design, and an operator defined by a sum over a Reynolds design a reductive Reynolds operator. For example, in the case of a graph with $n$ nodes, the computational complexity of the reductive Reynolds operator is reduced to $O(n^2)$, while the computational complexity of the Reynolds operator is $O(n! )$. We construct learning models based on the reductive Reynolds operator called equivariant and invariant Reynolds networks (ReyNets) and prove that they have universal approximation property. Reynolds designs for equivariant ReyNets are derived from combinatorial observations with Young diagrams, while Reynolds designs for invariant ReyNets are derived from invariants called Reynolds dimensions defined on the set of invariant polynomials. Numerical experiments show that the performance of our models is comparable to state-of-the-art methods.
Retention of Water in Terrestrial Magma Oceans and Carbon-rich Early Atmospheres

Massive steam and CO2 atmospheres have been proposed for magma ocean outgassing of Earth and terrestrial planets. Yet formation of such atmospheres depends on volatile exchange with the molten interior, governed by volatile solubilities and redox reactions. We determine the evolution of magma ocean–atmosphere systems for a range of oxygen fugacities, C/H ratios, and hydrogen budgets that include redox reactions for hydrogen (H2–H2O), carbon (CO–CO2), methane (CH4), and solubility laws for H2O and CO2. We find that small initial budgets of hydrogen, high C/H ratios, and oxidizing conditions suppress outgassing of hydrogen until the late stage of magma ocean crystallization. Hence, early atmospheres in equilibrium with magma oceans are dominantly carbon-rich, and specifically CO-rich except at the most oxidizing conditions. The high solubility of H2O limits its outgassing to melt fractions below ∼30%, the fraction at which the mantle transitions from vigorous to sluggish convection with melt percolation. Sluggish melt percolation could enable a surface lid to form, trapping water in the interior and thereby maintaining a carbon-rich atmosphere (equilibrium crystallization). Alternatively, efficient crystal settling could maintain a molten surface, promoting a transition to a water-rich atmosphere (fractional crystallization). However, additional processes, including melt trapping and H dissolution in crystallizing minerals, further conspire to limit the extent of H outgassing, even for fractional crystallization. Hence, much of the water delivered to planets during their accretion can be safely harbored in their interiors during the magma ocean stage, particularly at oxidizing conditions.
Scale-dependent inclination angle of turbulent structures in stratified atmospheric surface layers

Abstract A large-scale spanwise and wall-normal array of sonic anemometers in the atmospheric surface layer is used to acquire all three components of instantaneous fluctuating velocity as well as temperature in a range of stability conditions. These data permit investigation of the three-dimensional statistical structure of turbulence structures. Based on a similar dataset, Krug et al. (Boundary-Layer Meteorol., vol. 172, 2019, pp. 199–214) reported a self-similar range of wall-attached turbulence structures under both unstable and near-neutral stability conditions. They considered only a wall-normal array and thus assessed statistical structure in the wall-normal direction, in relation to the streamwise wavelength. The present work extends the view of a self-similar range of turbulence structures, by including the statistical structure in the spanwise direction. Moreover, by analysing the phase shift between synchronized measurements in the spectral domain, it is inferred how a scale-dependent inclination angle in the streamwise/wall-normal plane varies with stability. Results suggest that the self-similar wall-attached structures have similar aspect ratios between streamwise/wall-normal scales and streamwise/spanwise scales such that $\lambda _x/\Delta z : \lambda _x/\Delta y \approx 1$ for both near-neutral and unstable conditions. Under the most unstable conditions, coherent structures with $\lambda _x/\delta = 1$ are inclined at angles as high as $65^{\circ }$ relative to the solid boundary, while larger scales $\lambda _x/\delta = 6$ exhibit inclination angles of approximately $35^{\circ }$. For near-neutral stability conditions, the angle tends towards $12^{\circ }$ for all scales. It is noted that in the near-neutral condition, the structure inclination angle and the aspect ratio – and thus the statistical modelling of coherent structures in the atmospheric surface layer – are highly sensitive to the value of the stability parameter.
BBQ: A hand-built bias benchmark for question answering

It is well documented that NLP models learn social biases, but little work has been done on how these biases manifest in model outputs for applied tasks like question answering (QA). We introduce the Bias Benchmark for QA (BBQ), a dataset of question-sets constructed by the authors that highlight attested social biases against people belonging to protected classes along nine social dimensions relevant for U.S. English-speaking contexts. Our task evaluate model responses at two levels: (i) given an under-informative context, we test how strongly responses reflect social biases, and (ii) given an adequately informative context, we test whether the model’s biases override a correct answer choice. We find that models often rely on stereotypes when the context is under-informative, meaning the model’s outputs consistently reproduce harmful biases in this setting. Though models are more accurate when the context provides an informative answer, they still rely on stereotypes and average up to 3.4 percentage points higher accuracy when the correct answer aligns with a social bias than when it conflicts, with this difference widening to over 5 points on examples targeting gender for most models tested.
Distributed Computation of Stochastic GNE With Partial Information: An Augmented Best-Response Approach

In this article, we focus on the stochastic generalized Nash equilibrium problem (SGNEP) which is an important and widely used model in many different fields. In this model, subject to certain global resource constraints, a set of self-interested players aims to optimize their local objectives that depend on their own decisions and the decisions of others and are influenced by some random factors. We propose a distributed stochastic generalized Nash equilibrium seeking algorithm in a partial-decision information setting based on the Douglas–Rachford operator splitting scheme, which relaxes assumptions in the existing literature. The proposed algorithm updates players' local decisions through augmented best-response schemes and subsequent projections onto the local feasible sets, which comprise most of the computational workload. The projected stochastic subgradient method is applied to provide approximate solutions to the augmented best-response subproblems for each player. The Robbins–Siegmund theorem is leveraged to establish the main convergence results to a true Nash equilibrium using the proposed inexact solver. Finally, we illustrate the validity of the proposed algorithm via two numerical examples: 1) a stochastic Nash–Cournot distribution game and 2) a multiproduct assembly problem with the two-stage model.
Predicting Solar Flares with Remote Sensing and Machine Learning

High energy solar flares and coronal mass ejections have the potential to destroy Earth's ground and satellite infrastructures, causing trillions of dollars in damage and mass human suffering. Destruction of these critical systems would disable power grids and satellites, crippling communications and transportation. This would lead to food shortages and an inability to respond to emergencies. A solution to this impending problem is proposed herein using satellites in solar orbit that continuously monitor the Sun, use artificial intelligence and machine learning to calculate the probability of massive solar explosions from this sensed data, and then signal defense mechanisms that will mitigate the threat. With modern technology there may be only safeguards that can be implemented with enough warning, which is why the best algorithm must be identified and continuously trained with existing and new data to maximize true positive rates while minimizing false negatives. This paper conducts a survey of current machine learning models using open source solar flare prediction data. The rise of edge computing allows machine learning hardware to be placed on the same satellites as the sensor arrays, saving critical time by not having to transmit remote sensing data across the vast distances of space. A system of systems approach will allow enough warning for safety measures to be put into place mitigating the risk of disaster.
Physical activity at work may not be health enhancing. A systematic review with meta-analysis on the association between occupational physical activity and cardiovascular disease mortality covering 23 studies with 655 892 participants

Objectives Emerging evidence suggests contrasting health effects for leisure-time and occupational physical activity. In this systematic review, we synthesized and described the epidemiological evidence regarding the association between occupational physical activity and cardiovascular disease (CVD) mortality. Methods A literature search was performed in PubMed, Embase, CINAHL, PsycINFO and Evidence-Based Medicine Reviews, from database inception to 17 April 2020. Articles were included if they described original observational prospective research, assessing the association between occupational physical activity and CVD mortality among adult workers. Reviews were included if they controlled for age and gender and at least one other relevant variable. We performed meta-analyses on the associations between occupational physical activity and CVD mortality. Results We screened 3345 unique articles, and 31 articles (from 23 studies) were described in this review. In the meta-analysis, occupational physical activity showed no significant association with overall CVD mortality for both males [hazard ratio (HR) 1.00, 95% confidence interval (CI) 0.87–1.15] and females (HR 0.95, 95% CI 0.82–1.09). Additional analysis showed that higher levels of occupational physical activity were non-significantly associated with a 15% increase in studies reporting on the outcome ischemic heart disease mortality (HR 1.15, 95% CI 0.88–1.49). Conclusions While the beneficial association between leisure-time physical activity and CVD mortality has been widely documented, occupational physical activity was not found to have a beneficial association with CVD mortality. This observation may have implications for our appreciation of the association between physical activity and health for workers in physically demanding jobs, as occupational physical activity may not be health enhancing.
Pathogenesis-related protein 1 (PR-1) genes in soybean: genome-wide identification, structural analysis and expression profiling under multiple biotic and abiotic stresses

Plant pathogenesis-related (PR) proteins are a large group of proteins, classified in 17 families, that are induced by pathological conditions. Here, we characterized the soybean PR-1 (GmPR-1) gene repertoire at the sequence, structural and expression levels. We found 24 GmPR-1 genes, clustered in two phylogenetic groups. GmPR-1 genes are under strong purifying selection, particularly those that emerged by tandem duplications. GmPR-1 promoter regions are abundant in cis-regulatory elements associated with major stress-related transcription factor families, namely WRKY, ERF, HD-Zip, C2H2, NAC, and GATA. We observed that 23 GmPR-1 genes are induced by stress conditions or exclusively expressed upon stress. We explored 1972 transcriptome samples, including 26 stress conditions, revealing that most GmPR-1 genes are differentially expressed in a plethora of biotic and abiotic stresses. Our findings highlight stress-responsive GmPR-1 genes with potential biotechnological applications, such as the development of transgenic lines with increased resistance to biotic and abiotic stresses.
An Antigenically Diverse, Representative Panel of Envelope Glycoproteins for Hepatitis C Virus Vaccine Development

Background & Aims Development of a prophylactic hepatitis C virus (HCV) vaccine will require accurate and reproducible measurement of neutralizing breadth of vaccine-induced antibodies. Currently available HCV panels may not adequately represent the genetic and antigenic diversity of circulating HCV strains, and the lack of standardization of these panels makes it difficult to compare neutralization results obtained in different studies. Here, we describe the selection and validation of a genetically and antigenically diverse reference panel of 15 HCV pseudoparticles (HCVpps) for neutralization assays. Methods We chose 75 envelope (E1E2) clones to maximize representation of natural polymorphisms observed in circulating HCV isolates, and 65 of these clones generated functional HCVpps. Neutralization sensitivity of these HCVpps varied widely. HCVpps clustered into 15 distinct groups based on patterns of relative sensitivity to 7 broadly neutralizing monoclonal antibodies. We used these data to select a final panel of 15 antigenically representative HCVpps. Results Both the 65 and 15 HCVpp panels span 4 tiers of neutralization sensitivity, and neutralizing breadth measurements for 7 broadly neutralizing monoclonal antibodies were nearly equivalent using either panel. Differences in neutralization sensitivity between HCVpps were independent of genetic distances between E1E2 clones. Conclusions Neutralizing breadth of HCV antibodies should be defined using viruses spanning multiple tiers of neutralization sensitivity rather than panels selected solely for genetic diversity. We propose that this multitier reference panel could be adopted as a standard for the measurement of neutralizing antibody potency and breadth, facilitating meaningful comparisons of neutralization results from vaccine studies in different laboratories.