Miracle-dz/UniFAR-FTUs huggingface dataset:
Megapixel Image Generation with Step-Unrolled Denoising Autoencoders

An ongoing trend in generative modelling research has been to push sample resolutions higher whilst simultaneously reducing computational requirements for training and sampling. We aim to push this trend further via the combination of techniques - each component representing the current pinnacle of efficiency in their respective areas. These include vector-quantized GAN (VQ-GAN), a vector-quantization (VQ) model capable of high levels of lossy - but perceptually insignificant - compression; hourglass transformers, a highly scaleable self-attention model; and step-unrolled denoising autoencoders (SUNDAE), a non-autoregressive (NAR) text generative model. Unexpectedly, our method highlights weaknesses in the original formulation of hourglass transformers when applied to multidimensional data. In light of this, we propose modifications to the resampling mechanism, applicable in any task applying hierarchical transformers to multidimensional data. Additionally, we demonstrate the scalability of SUNDAE to long sequence lengths - four times longer than prior work. Our proposed framework scales to high-resolutions ($1024 \times 1024$) and trains quickly (2-4 days). Crucially, the trained model produces diverse and realistic megapixel samples in approximately 2 seconds on a consumer-grade GPU (GTX 1080Ti). In general, the framework is flexible: supporting an arbitrary number of sampling steps, sample-wise self-stopping, self-correction capabilities, conditional generation, and a NAR formulation that allows for arbitrary inpainting masks. We obtain FID scores of 10.56 on FFHQ256 - close to the original VQ-GAN in less than half the sampling steps - and 21.85 on FFHQ1024 in only 100 sampling steps.
The Origin of the Nuclear Star-forming Ring in NGC 3182

We investigate the stellar and ionized gas kinematics, and stellar populations of NGC 3182 galaxy using integral field spectrograph data from the Calar Alto Legacy Integral Field Area survey. We try to clarify the nature of the ring structure in NGC 3182. We find a negative stellar age gradient out to the ring, while [α/Fe] considerably enhanced in the ring. The stellar metallicity shows a smooth negative gradient. From the line-ratio diagnostic diagrams, we confirm that NGC 3182 is a Seyfert galaxy from emission-line flux ratio, while the gas in the inner ring is ionized mostly by young stars. However, any obvious feature of outflows is not found in its gas kinematics. In the ring, star formation seems to have recently occurred and the gas metallicity is slightly enhanced compared to the center. From our results, we conclude that star formation has occurred in the circumnuclear region within a short period and this may result from a positive feedback by active galactic nucleus radiation pressure.
BYOL-S: Learning Self-supervised Speech Representations by Bootstrapping

Methods for extracting audio and speech features have been studied since pioneering work on spectrum analysis decades ago. Recent efforts are guided by the ambition to develop general-purpose audio representations. For example, deep neural networks can extract optimal embeddings if they are trained on large audio datasets. This work extends existing methods based on self-supervised learning by bootstrapping, proposes various encoder architectures, and explores the effects of using different pre-training datasets. Lastly, we present a novel training framework to come up with a hybrid audio representation, which combines handcrafted and data-driven learned audio features. All the proposed representations were evaluated within the HEAR NeurIPS 2021 challenge for auditory scene classification and timestamp detection tasks. Our results indicate that the hybrid model with a convolutional transformer as the encoder yields superior performance in most HEAR challenge tasks.
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs

Despite their widespread success in various domains, Transformer networks have yet to perform well across datasets in the domain of 3D atomistic graphs such as molecules even when 3D-related inductive biases like translational invariance and rotational equivariance are considered. In this paper, we demonstrate that Transformers can generalize well to 3D atomistic graphs and present Equiformer, a graph neural network leveraging the strength of Transformer architectures and incorporating SE(3)/E(3)-equivariant features based on irreducible representations (irreps). First, we propose a simple and effective architecture by only replacing original operations in Transformers with their equivariant counterparts and including tensor products. Using equivariant operations enables encoding equivariant information in channels of irreps features without complicating graph structures. With minimal modifications to Transformers, this architecture has already achieved strong empirical results. Second, we propose a novel attention mechanism called equivariant graph attention, which improves upon typical attention in Transformers through replacing dot product attention with multi-layer perceptron attention and including non-linear message passing. With these two innovations, Equiformer achieves competitive results to previous models on QM9, MD17 and OC20 datasets.
ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings

We present a scalable approach for learning open-world object-goal navigation (ObjectNav) -- the task of asking a virtual robot (agent) to find any instance of an object in an unexplored environment (e.g.,"find a sink"). Our approach is entirely zero-shot -- i.e., it does not require ObjectNav rewards or demonstrations of any kind. Instead, we train on the image-goal navigation (ImageNav) task, in which agents find the location where a picture (i.e., goal image) was captured. Specifically, we encode goal images into a multimodal, semantic embedding space to enable training semantic-goal navigation (SemanticNav) agents at scale in unannotated 3D environments (e.g., HM3D). After training, SemanticNav agents can be instructed to find objects described in free-form natural language (e.g.,"sink","bathroom sink", etc.) by projecting language goals into the same multimodal, semantic embedding space. As a result, our approach enables open-world ObjectNav. We extensively evaluate our agents on three ObjectNav datasets (Gibson, HM3D, and MP3D) and observe absolute improvements in success of 4.2% - 20.0% over existing zero-shot methods. For reference, these gains are similar or better than the 5% improvement in success between the Habitat 2020 and 2021 ObjectNav challenge winners. In an open-world setting, we discover that our agents can generalize to compound instructions with a room explicitly mentioned (e.g.,"Find a kitchen sink") and when the target room can be inferred (e.g.,"Find a sink and a stove").
The curious case of GW200129: interplay between spin-precession inference and data-quality issues

Measurement of spin-precession in black hole binary mergers observed with gravitational waves is an exciting milestone as it relates to both general relativistic dynamics and astrophysical binary formation scenarios. In this study, we revisit the evidence for spin-precession in GW200129 and localize its origin to data in LIGO Livingston in the 20--50\,Hz frequency range where the signal amplitude is lower than expected from a non-precessing binary given all the other data. These data are subject to known data quality issues as a glitch was subtracted from the detector's strain data. The lack of evidence for spin-precession in LIGO Hanford leads to a noticeable inconsistency between the inferred binary mass ratio and precessing spin in the two LIGO detectors, something not expected from solely different Gaussian noise realizations. We revisit the LIGO Livingston glitch mitigation and show that the difference between a spin-precessing and a non-precessing interpretation for GW200129 is smaller than the statistical and systematic uncertainty of the glitch subtraction, finding that the support for spin-precession depends sensitively on the glitch modeling. We also investigate the signal-to-noise ratio $\sim7$ trigger in the less sensitive Virgo detector. Though not influencing the spin-precession studies, the Virgo trigger is grossly inconsistent with the ones in LIGO Hanford and LIGO Livingston as it points to a much heavier system. We interpret the Virgo data in the context of further data quality issues. While our results do not disprove the presence of spin-precession in GW200129, we argue that any such inference is contingent upon the statistical and systematic uncertainty of the glitch mitigation. Our study highlights the role of data quality investigations when inferring subtle effects such as spin-precession for short signals such as the ones produced by high-mass systems.
Learning Augmented Binary Search Trees

A treap is a classic randomized binary search tree data structure that is easy to implement and supports O (log n ) expected time access. However, classic treaps do not take advantage of the input distribution or patterns in the input. Given recent advances in algorithms with predictions, we propose pairing treaps with machine advice to form a learning-augmented treap. We are the first to propose a learning-augmented data structure that supports binary search tree operations such as range-query and successor functionalities. With the assumption that we have access to advice from a frequency estimation oracle, we assign learned priorities to the nodes to better improve the treap’s structure. We theoretically analyze the learning-augmented treap’s performance under various input distributions and show that under those circumstances, our learning-augmented treap has stronger guarantees than classic treaps and other classic tree-based data structures. Fur-ther, we experimentally evaluate our learned treap on synthetic datasets and demonstrate a performance advantage over other search tree data structures. We also present experiments on real world datasets with known frequency estimation oracles and show improvements as well.
SECLEDS: Sequence Clustering in Evolving Data Streams via Multiple Medoids and Medoid Voting

Sequence clustering in a streaming environment is challenging because it is computationally expensive, and the sequences may evolve over time. K-medoids or Partitioning Around Medoids (PAM) is commonly used to cluster sequences since it supports alignment-based distances, and the k-centers being actual data items helps with cluster interpretability. However, offline k-medoids has no support for concept drift, while also being prohibitively expensive for clustering data streams. We therefore propose SECLEDS, a streaming variant of the k-medoids algorithm with constant memory footprint. SECLEDS has two unique properties: i) it uses multiple medoids per cluster, producing stable high-quality clusters, and ii) it handles concept drift using an intuitive Medoid Voting scheme for approximating cluster distances. Unlike existing adaptive algorithms that create new clusters for new concepts, SECLEDS follows a fundamentally different approach, where the clusters themselves evolve with an evolving stream. Using real and synthetic datasets, we empirically demonstrate that SECLEDS produces high-quality clusters regardless of drift, stream size, data dimensionality, and number of clusters. We compare against three popular stream and batch clustering algorithms. The state-of-the-art BanditPAM is used as an offline benchmark. SECLEDS achieves comparable F1 score to BanditPAM while reducing the number of required distance computations by 83.7%. Importantly, SECLEDS outperforms all baselines by 138.7% when the stream contains drift. We also cluster real network traffic, and provide evidence that SECLEDS can support network bandwidths of up to 1.08 Gbps while using the (expensive) dynamic time warping distance.
Boson thermodynamics in the gravity field

A uniform force like the weight has been shown to forbid Bose-Einstein condensation (BEC), due to the discretness of the Airy spectrum, resulting from the weight. We show that BEC is forbidden even if the Airy spectrum is treated as continuous, in what we call the approximate continuous limit (ACL). The absence of BEC is due to the ﬁnite diﬀerence (cid:15) 0 between the quantum and the classical ground level. A thorough study is made, showing the way the Bose-Einstein condensate grows, at temperatures lower than the ﬁctiuos BEC temperature T f , resulting from the limit (cid:15) 0 → 0. A comparison with some experimental data shows the diﬀerence between the conﬁnement in a rigid-wall reservoir and in a harmonic trap.
Heterogeneous Target Speech Separation

We introduce a new paradigm for single-channel target source separation where the sources of interest can be distinguished using non-mutually exclusive concepts (e.g., loudness, gender, language, spatial location, etc). Our proposed heterogeneous separation framework can seamlessly leverage datasets with large distribution shifts and learn cross-domain representations under a variety of concepts used as conditioning. Our experiments show that training separation models with heterogeneous conditions facilitates the generalization to new concepts with unseen out-of-domain data while also performing substantially higher than single-domain specialist models. Notably, such training leads to more robust learning of new harder source separation discriminative concepts and can yield improvements over permutation invariant training with oracle source selection. We analyze the intrinsic behavior of source separation training with heterogeneous metadata and propose ways to alleviate emerging problems with challenging separation conditions. We release the collection of preparation recipes for all datasets used to further promote research towards this challenging task.
Self Supervised Learning for Few Shot Hyperspectral Image Classification

Deep learning has proven to be a very effective approach for Hyperspectral Image (HSI) classification. However, deep neural networks require large annotated datasets to generalize well. This limits the applicability of deep learning for HSI classification, where manually labelling thousands of pixels for every scene is impractical. In this paper, we propose to leverage Self Supervised Learning (SSL) for HSI classification. We show that by pre-training an encoder on unlabeled pixels using Barlow-Twins, a state-of-the-art SSL algorithm, we can obtain accurate models with a handful of labels. Experimental results demonstrate that this approach significantly outperforms vanilla supervised learning.
Geometry of Degeneracy in Potential and Density Space

In a previous work [J. Chem. Phys. 155, 244111 (2021)], we found counterexamples to the fundamental Hohenberg-Kohn theorem from density-functional theory in finite-lattice systems represented by graphs. Here, we demonstrate that this only occurs at very peculiar and rare densities, those where density sets arising from degenerate ground states, called degeneracy regions, touch each other or the boundary of the whole density domain. Degeneracy regions are shown to generally be in the shape of the convex hull of an algebraic variety, even in the continuum setting. The geometry arising between density regions and the potentials that create them is analyzed and explained with examples that, among other shapes, feature the Roman surface.
$\Lambda_b \to \Lambda_c$ Form Factors from QCD Light-Cone Sum Rules

In this work, we calculate the transition form factors of $\Lambda_b$ decaying into $\Lambda_c$ within the framework of light-cone sum rules with the distribution amplitudes (DAs) of $\Lambda_b$-baryon. In the hadronic representation of the correlation function, we have isolated both the $\Lambda_c$ and the $\Lambda_c^*$ states so that the $\Lambda_b \rightarrow \Lambda_c$ form factors can be obtained without ambiguity. We investigate the P-type and A-type current to interpolate the light baryons for a comparison since the interpolation current for the baryon state is not unique. We also employ three parametrization models for DAs of $\Lambda_b $ in the numerical calculation. We present the numerical predictions on the $\Lambda_b \rightarrow \Lambda_c$ form factors and the branching fractions, the averaged forward-backward asymmetry , the averaged final hadron polarization and the averaged lepton polarization of the $\Lambda_b \to \Lambda_c \ell\mu$ decays, as well as the ratio of branching ratios $R_{\Lambda_c}$, and the predicted $R_{\Lambda_c}$ can be consistent with the LHCb data.Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. Article funded by SCOAP3 and published under licence by Chinese Physical Society and the Institute of High Energy Physics of the Chinese Academy of Science and the Institute of Modern Physics of the Chinese Academy of Sciences and IOP Publishing Ltd.
Relative Survivable Network Design

One of the most important and well-studied settings for network design is edge-connectivity requirements. This encompasses uniform demands such as the Minimum k -Edge-Connected Spanning Subgraph problem ( k -ECSS), as well as nonuniform demands such as the Survivable Network Design problem. A weakness of these formulations, though, is that we are not able to ask for fault-tolerance larger than the connectivity. Taking inspiration from recent definitions and progress in graph spanners, we introduce and study new variants of these problems under a notion of relative fault-tolerance. Informally, we require not that two nodes are connected if there are a bounded number of faults (as in the classical setting), but that two nodes are connected if there are a bounded number of faults and the two nodes are connected in the underlying graph post-faults . That is, the subgraph we build must “behave” identically to the underlying graph with respect to connectivity after bounded faults. We define and introduce these problems, and provide the first approximation algorithms: a (1 + 4 /k )-approximation for the unweighted relative version of k -ECSS, a 2-approximation for the weighted relative version of k -ECSS, and a 27 / 4-approximation for the special case of Relative Survivable Network Design with only a single demand with a connectivity requirement of 3. To obtain these results, we introduce a number of technical ideas that may of independent interest. First, we give a generalization of Jain’s iterative rounding analysis that works even when the cut-requirement function is not weakly supermodular, but instead satisfies a weaker definition we introduce and term local weak supermodularity. Second, we prove a structure theorem and design an approximation algorithm utilizing a new decomposition based on important separators , which are structures commonly used in fixed-parameter algorithms that have not commonly been used in approximation algorithms.
A Flat-spectrum Radio Transient at 122 Mpc Consistent with an Emerging Pulsar Wind Nebula

We report the discovery and follow-up observations of VT 1137–0337, an unusual radio transient found in our systematic search for extragalactic explosions in the Very Large Array Sky Survey. It is located in the brightest region of a dwarf starburst galaxy at a luminosity distance of 121.6 Mpc. Its 3 GHz luminosity is comparable to luminous radio supernovae associated with dense circumstellar interaction and relativistic outflows. However, its broadband radio spectrum—proportional to ν −0.35 over a range of ≳10× in frequency and fading at a rate of 5% yr–1—cannot be directly explained by the shock of a stellar explosion. Jets launched by various classes of accreting black holes also struggle to account for VT 1137–0337's combination of observational properties. Instead, we propose that VT 1137–0337 is a decades-old pulsar wind nebula that has recently emerged from within the free–free opacity of its surrounding supernova ejecta. If the nebula is powered by spin-down, the central neutron star should have a surface dipole field of ∼1013–1014 G and a present-day spin period of ∼10–100 ms. Alternatively, the nebula may be powered by the release of magnetic energy from a magnetar. Magnetar nebulae have been proposed to explain the persistent radio sources associated with the repeating fast radio bursts FRB 121102 and FRB 190520B. These FRB persistent sources have not previously been observed as transients but do bear a striking resemblance to VT 1137–0337 in their radio luminosity, spectral index, and host galaxy properties.
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections

Linear time-invariant state space models (SSM) are a classical model from engineering and statistics, that have recently been shown to be very promising in machine learning through the Structured State Space sequence model (S4). A core component of S4 involves initializing the SSM state matrix to a particular matrix called a HiPPO matrix, which was empirically important for S4's ability to handle long sequences. However, the specific matrix that S4 uses was actually derived in previous work for a particular time-varying dynamical system, and the use of this matrix as a time-invariant SSM had no known mathematical interpretation. Consequently, the theoretical mechanism by which S4 models long-range dependencies actually remains unexplained. We derive a more general and intuitive formulation of the HiPPO framework, which provides a simple mathematical interpretation of S4 as a decomposition onto exponentially-warped Legendre polynomials, explaining its ability to capture long dependencies. Our generalization introduces a theoretically rich class of SSMs that also lets us derive more intuitive S4 variants for other bases such as the Fourier basis, and explains other aspects of training S4, such as how to initialize the important timescale parameter. These insights improve S4's performance to 86% on the Long Range Arena benchmark, with 96% on the most difficult Path-X task.
The Properties of Fast Yellow Pulsating Supergiants: FYPS Point the Way to Missing Red Supergiants

Fast yellow pulsating supergiants (FYPS) are a recently discovered class of evolved massive pulsators. As candidate supergiant objects, and one of the few classes of pulsating evolved massive stars, these objects have incredible potential to change our understanding of the structure and evolution of massive stars. Here we examine the lightcurves of a sample of 126 cool supergiants in the Magellanic Clouds observed by the Transiting Exoplanet Survey Satellite in order to identify pulsating stars. After making quality cuts and filtering out contaminant objects, we examine the distribution of pulsating stars in the Hertzprung–Russel (HR) diagram, and find that FYPS occupy a region above logL/L⊙≳5.0 . This luminosity boundary corresponds to stars with initial masses of ∼18–20 M ⊙, consistent with the most massive red supergiant progenitors of supernovae (SNe) II-P, as well as the observed properties of SNe IIb progenitors. This threshold is in agreement with the picture that FYPS are post-RSG stars. Finally, we characterize the behavior of FYPS pulsations as a function of their location in the HR diagram. We find low-frequency pulsations at higher effective temperatures, and higher-frequency pulsations at lower temperatures, with a transition between the two behaviors at intermediate temperatures. The observed properties of FYPS make them fascinating objects for future theoretical study.
COUPLED FIXED POINT THEOREM FOR HYBRID PAIRS OF MAPPINGS UNDER ϕ − ψ CONTRACTION IN FUZZY METRIC SPACES

In this paper, we establish some common coupled ﬁxed point theorems for two hybrid pairs of mappings under ϕ − ψ contraction on noncomplete fuzzy metric spaces using the property ( EA ) introduced by Deshpande and Handa [3]. We improve, extend and generalize the results of Deshpande and Handa [3] and several other known results in metric spaces to fuzzy metric spaces.