Miracle-dz/UniFAR-FTUs huggingface dataset:
Coherent Reconstruction of Multiple Humans From a Single Image

In this work, we address the problem of multi-person 3D pose estimation from a single image. A typical regression approach in the top-down setting of this problem would first detect all humans and then reconstruct each one of them independently. However, this type of prediction suffers from incoherent results, e.g., interpenetration and inconsistent depth ordering between the people in the scene. Our goal is to train a single network that learns to avoid these problems and generate a coherent 3D reconstruction of all the humans in the scene. To this end, a key design choice is the incorporation of the SMPL parametric body model in our top-down framework, which enables the use of two novel losses. First, a distance field-based collision loss penalizes interpenetration among the reconstructed people. Second, a depth ordering-aware loss reasons about occlusions and promotes a depth ordering of people that leads to a rendering which is consistent with the annotated instance segmentation. This provides depth supervision signals to the network, even if the image has no explicit 3D annotations. The experiments show that our approach outperforms previous methods on standard 3D pose benchmarks, while our proposed losses enable more coherent reconstruction in natural images. The project website with videos, results, and code can be found at: https://jiangwenpl.github. io/multiperson
Entropic gradient descent algorithms and wide flat minima

The properties of flat minima in the empirical risk landscape of neural networks have been debated for some time. Increasing evidence suggests they possess better generalization capabilities with respect to sharp ones. In this work we first discuss the relationship between alternative measures of flatness: the local entropy, which is useful for analysis and algorithm development, and the local energy, which is easier to compute and was shown empirically in extensive tests on state-of-the-art networks to be the best predictor of generalization capabilities. We show semi-analytically in simple controlled scenarios that these two measures correlate strongly with each other and with generalization. Then, we extend the analysis to the deep learning scenario by extensive numerical validations. We study two algorithms, entropy-stochastic gradient descent and replicated-stochastic gradient descent, that explicitly include the local entropy in the optimization objective. We devise a training schedule by which we consistently find flatter minima (using both flatness measures), and improve the generalization error for common architectures (e.g. ResNet, EfficientNet).
The Pitfalls of Simplicity Bias in Neural Networks

Several works have proposed Simplicity Bias (SB)---the tendency of standard training procedures such as Stochastic Gradient Descent (SGD) to find simple models---to justify why neural networks generalize well [Arpit et al. 2017, Nakkiran et al. 2019, Valle-Perez et al. 2019]. However, the precise notion of simplicity remains vague. Furthermore, previous settings that use SB to justify why neural networks generalize well do not simultaneously capture the brittleness of neural networks---a widely observed phenomenon in practice [Goodfellow et al. 2014, Jo and Bengio 2017]. To this end, we introduce a collection of piecewise-linear and image-based datasets that (a) naturally incorporate a precise notion of simplicity and (b) capture the subtleties of neural networks trained on real datasets. Through theory and experiments on these datasets, we show that SB of SGD and variants is extreme: neural networks rely exclusively on the simplest feature and remain invariant to all predictive complex features. Consequently, the extreme nature of SB explains why seemingly benign distribution shifts and small adversarial perturbations significantly degrade model performance. Moreover, contrary to conventional wisdom, SB can also hurt generalization on the same data distribution, as SB persists even when the simplest feature has less predictive power than the more complex features. We also demonstrate that common approaches for improving generalization and robustness---ensembles and adversarial training---do not mitigate SB and its shortcomings. Given the central role played by SB in generalization and robustness, we hope that the datasets and methods in this paper serve as an effective testbed to evaluate novel algorithmic approaches aimed at avoiding the pitfalls of extreme SB.
Improved Conditional Flow Models for Molecule to Image Synthesis

In this paper, we aim to synthesize cell microscopy images under different molecular interventions, motivated by practical applications to drug development. Building on the recent success of graph neural networks for learning molecular embeddings and flow-based models for image generation, we propose Mol2Image: a flow-based generative model for molecule to cell image synthesis. To generate cell features at different resolutions and scale to high-resolution images, we develop a novel multi-scale flow architecture based on a Haar wavelet image pyramid. To maximize the mutual information between the generated images and the molecular interventions, we devise a training strategy based on contrastive learning. To evaluate our model, we propose a new set of metrics for biological image generation that are robust, interpretable, and relevant to practitioners. We show quantitatively that our method learns a meaningful embedding of the molecular intervention, which is translated into an image representation reflecting the biological effects of the intervention.
Temporal Variability in Implicit Online Learning

In the setting of online learning, Implicit algorithms turn out to be highly successful from a practical standpoint. However, the tightest regret analyses only show marginal improvements over Online Mirror Descent. In this work, we shed light on this behavior carrying out a careful regret analysis. We prove a novel static regret bound that depends on the temporal variability of the sequence of loss functions, a quantity which is often encountered when considering dynamic competitors. We show, for example, that the regret can be constant if the temporal variability is constant and the learning rate is tuned appropriately, without the need of smooth losses. Moreover, we present an adaptive algorithm that achieves this regret bound without prior knowledge of the temporal variability and prove a matching lower bound. Finally, we validate our theoretical findings on classification and regression datasets.
Chiral anomaly and the pion properties in the light-front quark model

We explore the link between the chiral symmetry of QCD and the numerical results of the light-front quark model, analyzing both the two-point and three-point functions of the pion. Including the axial-vector coupling as well as the pseudoscalar coupling in the light-front quark model, we discuss the implication of the chiral anomaly in describing the pion decay constant, the pion-photon transition form factor and the electromagnetic form factor of the pion. In constraining the model parameters, we find that the chiral anomaly plays a critical role and the analysis of $F_{\pi\gamma}(Q^2)$ in timelike region is important. Our results indicate that the constituent quark picture is effective for the low and high $Q^2$ ranges implementing the quark mass evolution effect as $Q^2$ grows.
LaRVAE: Label Replacement VAE for Semi-Supervised Disentanglement Learning

Learning interpretable and disentangled representations is a crucial yet challenging task in representation learning. In this work, we develop LaRVAE, a novel semi-supervised variational auto-encoder (VAE) for learning disentangled representations by more effectively exploiting the limited labeled data. Speciﬁcally, during training, we replace the inferred representation associated with a data point with its ground-truth representation whenever it is available. LaRVAE is theoretically inspired by our proposed general framework of semi-supervised disentanglement learning in the context of VAEs. It naturally motivates both the novel representation replacement strategy in LaRVAE and a supervised regularization term commonly (albeit in an ad-hoc way) used in existing semi-supervised VAEs. Extensive experiments on synthetic and real datasets demonstrate both quantitatively and qualitatively the ability of LaRVAE to signiﬁcantly and consistently improve disentanglement with very limited supervision.
Exploiting the ConvLSTM: Human Action Recognition using Raw Depth Video-Based Recurrent Neural Networks

As in many other different fields, deep learning has become the main approach in most computer vision applications, such as scene understanding, object recognition, computer-human interaction or human action recognition (HAR). Research efforts within HAR have mainly focused on how to efficiently extract and process both spatial and temporal dependencies of video sequences. In this paper, we propose and compare, two neural networks based on the convolutional long short-term memory unit, namely ConvLSTM, with differences in the architecture and the long-term learning strategy. The former uses a video-length adaptive input data generator (\emph{stateless}) whereas the latter explores the \emph{stateful} ability of general recurrent neural networks but applied in the particular case of HAR. This stateful property allows the model to accumulate discriminative patterns from previous frames without compromising computer memory. Experimental results on the large-scale NTU RGB+D dataset show that the proposed models achieve competitive recognition accuracies with lower computational cost compared with state-of-the-art methods and prove that, in the particular case of videos, the rarely-used stateful mode of recurrent neural networks significantly improves the accuracy obtained with the standard mode. The recognition accuracies obtained are 75.26\% (CS) and 75.45\% (CV) for the stateless model, with an average time consumption per video of 0.21 s, and 80.43\% (CS) and 79.91\%(CV) with 0.89 s for the stateful version.
The R-Process Alliance: A Very Metal-poor, Extremely r-process-enhanced Star with [Eu/Fe] = + 2.2, and the Class of r-III Stars

We report the discovery of J1521−3538, a bright (V = 12.2), very metal-poor ([Fe/H] = −2.8) strongly r-process-enhanced field horizontal branch star, based on a high-resolution, high signal-to-noise Magellan/MIKE spectrum. J1521−3538 shows the largest r-process element overabundance in any known r-process-enhanced star, with [Eu/Fe] = +2.2, and its chemical abundances of 22 neutron-capture elements closely match the scaled solar r-process pattern. J1521−3538 is also one of few known carbon-enhanced metal-poor stars with r-process enhancement (CEMP-r stars), as found after correcting the measured C abundance for the star’s evolutionary status. We propose to extend the existing classification of moderately enhanced ( ) r-I and strongly r-process enhanced ( ) r-II stars to include an r-III class, for r-process stars such as J1521−3538, with and , or times the solar ratio of europium to iron. Using cosmochronometry, we estimate J1521−3538 to be and , using two different sets of initial production ratios. These ages are based on measurements of the Th line at 4019 Å and other r-process element abundances. This is broadly consistent with the old age of a low-mass, metal-poor field red horizontal branch star. J1521−3538 likely originated in a low-mass dwarf galaxy that was later accreted by the Milky Way, as evidenced by its highly eccentric orbit.
Optical and X-ray study of the peculiar high-mass X-ray binary XMMU J010331.7−730144

For a long time XMMU J010331.7−730144 was proposed as a high-mass X-ray binary candidate based on its X-ray properties, however, its optical behaviour was unclear – in particular previous observations did not reveal key Balmer emission lines. In this paper, we report on optical and X-ray variability of the system. XMMU J010331.7–730144 has been monitored with the Optical Gravitational Lensing Experiment (OGLE) in the I and V bands for the past 9 yr where it has shown extremely large amplitude outbursts separated by long periods of low-level flux. During its most recent optical outburst we obtained spectra with the Southern African Large Telescope (SALT) where, for the first time, the H α line is seen in emission, confirming the Be nature of the optical companion. The OGLE colour–magnitude diagrams also exhibit a distinct loop that is explained by changes in mass-loss from the Be star and mass outflow in its disc. In the X-rays, XMMU J010331.7−730144 has been monitored by the Neil Gehrels Swift Observatory through the S-CUBED programme. The X-ray flux throughout the monitoring campaign shows relatively low values for a typical Be/X-ray binary system. We show, from the analysis of the optical data, that the variability is due to the Be disc density and opacity changing rather than its physical extent as a result of efficient truncation by the NS. The relatively low X-ray flux can then be explained by the neutron star normally accreting matter at a low rate due to the small radial extent of the Be disc.
The formation of a blue cluster in the local Universe

It is well known from the Butcher-Oemler effect that galaxies in dense environment are mostly red with little star formation and the fraction of blue galaxies in galaxy groups/clusters also declines rapidly with redshifts. A recent work by Hashimoto et al. reported a local 'blue cluster' with high fraction of blue galaxies ($\sim 0.57$), higher than the model predictions. They ascribed this blue cluster to the feeding of gas along a filamentary structure around the cluster. In this work we use group catalog from the Sloan Digital Sky Survey Data Release 7 (SDSS DR7) and the state-of-art of semi-analytic model (SAM) to investigate the formation of blue clusters in local Universe. In total, we find four blue clusters with halo mass $\sim 10^{14}M_{\odot}$ at $0.02 < z < 0.082$, while only the one found by Hashimoto et al. is in a filamentary structure. The SAM predicts that blue clusters have later formation time and most blue satellite galaxies are recently accreted. We conclude that the formation of blue clusters is mainly governed by newly accreted blue satellites, rather than the effect of large-scale environment.
A massive mess: When a large dwarf and a Milky Way-like galaxy merge

Circa 10 billion years ago the Milky Way merged with a massive satellite, Gaia-Enceladus. To gain insight into the properties of its debris we analyse in detail the suite of simulations from Villalobos & Helmi (2008), which includes an experiment that produces a good match to the kinematics of nearby halo stars inferred from Gaia data. We compare the kinematic distributions of stellar particles in the simulations and study the distribution of debris in orbital angular momentum, eccentricity and energy, and its relation to the mass-loss history of the simulated satellite. We confirm that Gaia-Enceladus probably fell in on a retrograde, 30$^\circ$ inclination orbit. We find that while 75% of the debris in our preferred simulation has large eccentricity ($> 0.8$), roughly 9% has eccentricity smaller than 0.6. Star particles lost early have large retrograde motions, and a subset of these have low eccentricity. Such stars would be expected to have lower metallicities as they stem from the outskirts of the satellite, and hence naively they could be confused with debris associated with a separate system. These considerations seem to apply to some of the stars from the postulated Sequoia galaxy. When a massive discy galaxy merges, it leaves behind debris with a complex phase-space structure, a large range of orbital properties, and a range of chemical abundances. Observationally, this results in substructures with very different properties, which can be misinterpreted as implying independent progeny. Detailed chemical abundances of large samples of stars and tailored hydrodynamical simulations are critical to resolving such conundrums.
Revisiting nonfactorizable contributions to factorization-forbidden decays of B mesons to charmonium

Motivated by the large rates of $B\rightarrow (\chi_{c0}, \chi_{c2}, h_c)K$ decays observed by the $BABAR$ and Belle collaborations, we investigate the nonfactorizable contributions to these factorization-forbidden decays, which can occur through a gluon exchange between the $c\bar c$ system and the spectator quark. Our numerical results demonstrate that the spectator contributions are capable of producing a large branching ratio consistent with the experiments. As a by-product, we also study the Cabibbo-suppressed decays, such as $B\rightarrow (\chi_{c0}, \chi_{c2}, h_c)\pi$ and the U-spin-related $B_s$ decay, which have so far received less theoretical and experimental attention. The calculated branching ratios reach the order of $10^{-6}$, which in within the scope of the Belle-II and LHCb experiments. Further, the $CP$-asymmetry parameters are also calculated for these decays. The obtained results are compared with the available experimental data and numbers from other predictions. We also investigate the sources of theoretical uncertainties in our calculation.
The Reflectron: Exploiting geometry for learning generalized linear models

Generalized linear models (GLMs) extend linear regression by generating the dependent variables through a nonlinear function of a predictor in a Reproducing Kernel Hilbert Space. Despite nonconvexity of the underlying optimization problem, the GLM-tron algorithm of Kakade et al. (2011) provably learns GLMs with guarantees of computational and statistical efficiency. We present an extension of the GLM-tron to a mirror descent or natural gradient-like setting, which we call the Reflectron. The Reflectron enjoys the same statistical guarantees as the GLM-tron for any choice of the convex potential function $\psi$ used to define mirror descent. Central to our algorithm, $\psi$ can be chosen to implicitly regularize the learned model when there are multiple hypotheses consistent with the data. Our results extend to the case of multiple outputs with or without weight sharing. We perform our analysis in continuous-time, leading to simple and intuitive derivations, with discrete-time implementations obtained by discretization of the continuous-time dynamics. We supplement our theoretical analysis with simulations on real and synthetic datasets demonstrating the validity of our theoretical results.
Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Games

Finding approximate Nash equilibria in zero-sum imperfect-information games is challenging when the number of information states is large. Policy Space Response Oracles (PSRO) is a deep reinforcement learning algorithm grounded in game theory that is guaranteed to converge to an approximate Nash equilibrium. However, PSRO requires training a reinforcement learning policy at each iteration, making it too slow for large games. We show through counterexamples and experiments that DCH and Rectified PSRO, two existing approaches to scaling up PSRO, fail to converge even in small games. We introduce Pipeline PSRO (P2SRO), the first scalable general method for finding approximate Nash equilibria in large zero-sum imperfect-information games. P2SRO is able to parallelize PSRO with convergence guarantees by maintaining a hierarchical pipeline of reinforcement learning workers, each training against the policies generated by lower levels in the hierarchy. We show that unlike existing methods, P2SRO converges to an approximate Nash equilibrium, and does so faster as the number of parallel workers increases, across a variety of imperfect information games. We also introduce an open-source environment for Barrage Stratego, a variant of Stratego with an approximate game tree complexity of $10^{50}$. P2SRO is able to achieve state-of-the-art performance on Barrage Stratego and beats all existing bots.
Cold molecular gas and free–free emission from hot, dust-obscured galaxies at z ∼ 3

We report on observations of redshifted CO(1-0) line emission and observed-frame $\rm\sim$ 30GHz radio continuum emission from five ultra-luminous, mid-IR selected hot, Dust-Obscured Galaxies (Hot DOGs) at $z\rm\gtrsim$ 3 using the Karl G. Jansky Very Large Array. We detect CO(1-0) line emission in all five Hot DOGs, with one of them at high signal to noise. We analyse FIR-radio spectral energy distributions, including dust, free-free and synchrotron emission for the galaxies. We find that most of the 115 GHz rest-frame continuum is mostly due to synchrotron or free-free emission, with only a potentially small contribution from thermal emission. We see a deficit in the rest-frame 115 GHz continuum emission compared to dusty star-forming galaxies (DSFGs) and sub-millimetre galaxies (SMGs) at high redshift, suggesting that Hot DOGs do not have similar cold gas reserves compared with star-forming galaxies. One target, W2305-0039, is detected in the FIRST 1.4 GHz survey, and is likely to possess compact radio jets. We compare to the FIR-radio correlation, and find that at least half of the Hot DOGs in our sample are radio-quiet with respect to normal galaxies. These findings suggest that Hot DOGs have comparably less cold molecular gas than star-forming galaxies at lower, $z\rm\sim$ 2 redshifts, and are dominated by powerful, yet radio-quiet AGN.
Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energy

In this paper, we present a general method that can improve the sample quality of pre-trained likelihood based generative models. Our method constructs an energy function on the latent variable space that yields an energy function on samples produced by the pre-trained generative model. The energy based model is efficiently trained by maximizing the data likelihood, and after training, new samples in the latent space are generated from the energy based model and passed through the generator to producing samples in observation space. We show that using our proposed method, we can greatly improve the sample quality of popular likelihood based generative models, such as normalizing flows and VAEs, with very little computational overhead.
Algorithmically Optimal Outer Measures

We investigate the relationship between algorithmic fractal dimensions and the classical local fractal dimensions of outer measures in Euclidean spaces. We introduce global and local optimality conditions for lower semicomputable outer measures. We prove that globally optimal outer measures exist. Our main theorem states that the classical local fractal dimensions of any locally optimal outer measure coincide exactly with the algorithmic fractal dimensions. Our proof uses an especially convenient locally optimal outer measure $\boldsymbol{\kappa}$ defined in terms of Kolmogorov complexity. We discuss implications for point-to-set principles.
On the Expressivity of Neural Networks for Deep Reinforcement Learning

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-functions and policies are much more complex than the dynamics. We hypothesize many real-world MDPs also have a similar property. For these MDPs, model-based planning is a favorable algorithm, because the resulting policies can approximate the optimal policy significantly better than a neural network parameterization can, and model-free or model-based policy optimization rely on policy parameterization. Motivated by the theory, we apply a simple multi-step model-based bootstrapping planner (BOOTS) to bootstrap a weak $Q$-function into a stronger policy. Empirical results show that applying BOOTS on top of model-based or model-free policy optimization algorithms at the test time improves the performance on MuJoCo benchmark tasks.