Miracle-dz/UniFAR-FTUs huggingface dataset:
Novel View Synthesis with Diffusion Models

We present 3DiM, a diffusion model for 3D novel view synthesis, which is able to translate a single input view into consistent and sharp completions across many views. The core component of 3DiM is a pose-conditional image-to-image diffusion model, which takes a source view and its pose as inputs, and generates a novel view for a target pose as output. 3DiM can generate multiple views that are 3D consistent using a novel technique called stochastic conditioning. The output views are generated autoregressively, and during the generation of each novel view, one selects a random conditioning view from the set of available views at each denoising step. We demonstrate that stochastic conditioning significantly improves the 3D consistency of a naive sampler for an image-to-image diffusion model, which involves conditioning on a single fixed view. We compare 3DiM to prior work on the SRN ShapeNet dataset, demonstrating that 3DiM's generated completions from a single view achieve much higher fidelity, while being approximately 3D consistent. We also introduce a new evaluation methodology, 3D consistency scoring, to measure the 3D consistency of a generated object by training a neural field on the model's output views. 3DiM is geometry free, does not rely on hyper-networks or test-time optimization for novel view synthesis, and allows a single model to easily scale to a large number of scenes.
Grow and Merge: A Unified Framework for Continuous Categories Discovery

Although a number of studies are devoted to novel category discovery, most of them assume a static setting where both labeled and unlabeled data are given at once for finding new categories. In this work, we focus on the application scenarios where unlabeled data are continuously fed into the category discovery system. We refer to it as the {\bf Continuous Category Discovery} ({\bf CCD}) problem, which is significantly more challenging than the static setting. A common challenge faced by novel category discovery is that different sets of features are needed for classification and category discovery: class discriminative features are preferred for classification, while rich and diverse features are more suitable for new category mining. This challenge becomes more severe for dynamic setting as the system is asked to deliver good performance for known classes over time, and at the same time continuously discover new classes from unlabeled data. To address this challenge, we develop a framework of {\bf Grow and Merge} ({\bf GM}) that works by alternating between a growing phase and a merging phase: in the growing phase, it increases the diversity of features through a continuous self-supervised learning for effective category mining, and in the merging phase, it merges the grown model with a static one to ensure satisfying performance for known classes. Our extensive studies verify that the proposed GM framework is significantly more effective than the state-of-the-art approaches for continuous category discovery.
Multidimensional probability inequalities via spherical symmetry

Spherical symmetry arguments are used to produce a general device to convert identities and inequalities for the p th absolute moments of real-valued random variables into the corresponding identities and inequalities for the p th moments of the norms of random vectors in Hilbert spaces. Partic- ular results include the following: (i) an expression of the p th moment of the norm of such a random vector X in terms of the characteristic functional of X ; (ii) an extension of a previously obtained von Bahr–Esseen-type inequality for real-valued random variables with the best possible constant factor to random vectors in Hilbert spaces, still with the best possible constant factor; (iii) an extension of a previously obtained inequality between measures of “contrast between populations” and “spread within populations” to random vectors in Hilbert spaces.
UL2: Unifying Language Learning Paradigms

Existing pre-trained models are generally geared towards a particular class of problems. To date, there seems to be still no consensus on what the right architecture and pre-training setup should be. This paper presents a unified framework for pre-training models that are universally effective across datasets and setups. We begin by disentangling architectural archetypes with pre-training objectives -- two concepts that are commonly conflated. Next, we present a generalized&unified perspective for self-supervision in NLP and show how different pre-training objectives can be cast as one another and how interpolating between different objectives can be effective. We then propose Mixture-of-Denoisers (MoD), a pre-training objective that combines diverse pre-training paradigms together. We furthermore introduce a notion of mode switching, wherein downstream fine-tuning is associated with specific pre-training schemes. We conduct extensive ablative experiments to compare multiple pre-training objectives and find that our method pushes the Pareto-frontier by outperforming T5&GPT-like models across multiple diverse setups. By scaling our model up to 20B parameters, we achieve SOTA performance on 50 well-established supervised finetuning based NLP tasks. Our model also achieve strong results at in-context learning, outperforming 175B GPT-3 on zero-shot SuperGLUE and tripling the performance of T5-XXL on one-shot summarization. On 0-shot MMLU, UL2 20B outperforms T0 and T5 models. UL2 20B also works well with chain-of-thought prompting and reasoning, making it an appealing choice for research into reasoning at a small to medium scale of 20B parameters. Finally, we apply FLAN instruction tuning to the UL2 20B model, achieving MMLU and Big-Bench scores competitive to FLAN-PaLM 62B. We release Flax-based T5X checkpoints for the UL2 20B&Flan-UL2 20B.
A Behavior Regularized Implicit Policy for Offline Reinforcement Learning

Offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. The lack of environmental interactions makes the policy training vulnerable to state-action pairs far from the training dataset and prone to missing rewarding actions. For training more effective agents, we propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. We further propose a simple modification to the classical policy-matching methods for regularizing with respect to the dual form of the Jensen--Shannon divergence and the integral probability metrics. We theoretically show the correctness of the policy-matching approach, and the correctness and a good finite-sample property of our modification. An effective instantiation of our framework through the GAN structure is provided, together with techniques to explicitly smooth the state-action mapping for robust generalization beyond the static dataset. Extensive experiments and ablation study on the D4RL benchmark validate our framework and the effectiveness of our algorithmic designs.
ConfLab: A Data Collection Concept, Dataset, and Benchmark for Machine Analysis of Free-Standing Social Interactions in the Wild

Recording the dynamics of unscripted human interactions in the wild is challenging due to the delicate trade-offs between several factors: participant privacy, ecological validity, data fidelity, and logistical overheads. To address these, following a 'datasets for the community by the community' ethos, we propose the Conference Living Lab (ConfLab): a new concept for multimodal multisensor data collection of in-the-wild free-standing social conversations. For the first instantiation of ConfLab described here, we organized a real-life professional networking event at a major international conference. Involving 48 conference attendees, the dataset captures a diverse mix of status, acquaintance, and networking motivations. Our capture setup improves upon the data fidelity of prior in-the-wild datasets while retaining privacy sensitivity: 8 videos (1920x1080, 60 fps) from a non-invasive overhead view, and custom wearable sensors with onboard recording of body motion (full 9-axis IMU), privacy-preserving low-frequency audio (1250 Hz), and Bluetooth-based proximity. Additionally, we developed custom solutions for distributed hardware synchronization at acquisition and time-efficient continuous annotation of body keypoints and actions at high sampling rates. Our benchmarks showcase some of the open research tasks related to in-the-wild privacy-preserving social data analysis: keypoints detection from overhead camera views, skeleton-based no-audio speaker detection, and F-formation detection.
Breakdown of topological protection due to nonmagnetic edge disorder in two-dimensional materials in the quantum spin Hall phase

We study the suppression of the conductance quantization in quantum spin Hall systems by a combined eﬀect of electronic interactions and edge disorder, that is ubiquitous in exfoliated and CVD grown 2D materials. We show that the interplay between the electronic localized states due to edge defects and electron-electron interactions gives rise to local magnetic moments, that break time-reversal symmetry and the topological protection of the edge states in 2D topological systems. Our results suggest that edge disorder leads to small deviations of a perfect quantized conductance in short samples and to a strong conductance suppression in long ones. Our analysis is based on on the Kane-Mele model, an unrestricted Hubbard mean ﬁeld Hamiltonian and on a self-consistent recursive Green’s functions technique to calculate the transport quantities.
Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization

Warning: this paper contains model outputs exhibiting offensiveness and biases. Recently pre-trained language models (PLMs) have prospered in various natural language generation (NLG) tasks due to their ability to generate fairly fluent text. Nevertheless, these models are observed to capture and reproduce harmful contents in training corpora, typically toxic language and social biases, raising severe moral issues. Prior works on ethical NLG tackle detoxifying and debiasing separately, which is problematic since we find debiased models still exhibit toxicity while detoxified ones even exacerbate social biases. To address such a challenge, we propose the first unified framework of detoxifying and debiasing called UDDIA, which jointly formalizes these two problems as rectifying the output space. We theoretically interpret our framework as learning a text distribution mixing weighted attributes. Besides, UDDIA conducts adaptive optimization of only a few parameters during decoding based on a parameter-efficient tuning schema without any training data. This leads to minimal generation quality loss and improved rectification performance with acceptable computational cost. Experimental results demonstrate that compared to several strong baselines, UDDIA achieves debiasing and detoxifying simultaneously and better balances efficiency and effectiveness, taking a further step towards practical ethical NLG.
Super-resolution imaging with radio interferometry using sparse modeling

We propose a new technique to obtain super-resolution images with radio interferometer using sparse modeling. In standard radio interferometry, sampling of ($u$, $v$) is quite often incomplete and thus obtaining an image from observed visibilities becomes an underdetermined problem, and a technique so-called "zero-padding" is often used to fill up unsampled grids in ($u$, $v$) plane, resulting in image degradation by finite beam size as well as numerous side-lobes. In this paper we show that directly solving such an underdetermined problem based on sparse modeling (in this paper LASSO) avoids the above problems introduced by zero-padding, leading to super-resolution images in which structure finer than the standard beam size (diffraction limit) can be reproduced. We present results of one-dimensional and two-dimensional simulations of interferometric imaging, and discuss its implications to super-resolution imaging, particularly focusing on imaging of black hole shadows with millimeter VLBI.
Mapping the Most Massive Overdensities through Hydrogen (MAMMOTH). II. Discovery of the Extremely Massive Overdensity BOSS1441 at z = 2.32

Cosmological simulations suggest a strong correlation between high optical-depth Lyα absorbers, which arise from the intergalactic medium, and 3D mass overdensities on scales of 10–30 comoving Mpc. By examining the absorption spectra of ∼80,000 QSO sight lines over a volume of 0.1 Gpc3 in the Sloan Digital Sky Survey III (SDSS-III), we have identified an extreme overdensity, BOSS1441, which contains a rare group of strong Lyα absorbers at . This absorber group is associated with six QSOs at the same redshift on a 30 comoving Mpc scale. Using Mayall/MOSAIC narrowband and broadband imaging, we detect Lyα emitters (LAEs) down to and reveal a large-scale structure of LAEs in this field. Our follow-up LBT observations have spectroscopically confirmed 20 galaxies in the density peak. We show that BOSS1441 has an LAE overdensity of 10.8 ± 2.6 on a 15 comoving Mpc scale, which could collapse to a massive cluster with M ≳ 1015 at . This overdensity is among the most massive large-scale structures at discovered to date.
The Beltrami Equation: A Geometric Approach

1. Introduction. - 2. Preliminaries. - 3. The Classical Beltrami Equation ||mu|| < 1. - 4. The Degenerate Case. - 5. BMO- and FMO-Quasiconformal Mappings. - 6. Ring Q-Homeomorphisms at Boundary Points. - 7. Strong Ring Solutions of Beltrami Equations. - 8. On the Dirichlet Problem for Beltrami Equations. - 9. On the Beltrami Equations with Two Characteristics. - 10. Alternating Beltrami Equation. - References. - Index.
Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs

Transformer models have made tremendous progress in various fields in recent years. In the field of computer vision, vision transformers (ViTs) also become strong alternatives to convolutional neural networks (ConvNets), yet they have not been able to replace ConvNets since both have their own merits. For instance, ViTs are good at extracting global features with attention mechanisms while ConvNets are more efficient in modeling local relationships due to their strong inductive bias. A natural idea that arises is to combine the strengths of both ConvNets and ViTs to design new structures. In this paper, we propose a new basic neural network operator named position-aware circular convolution (ParC) and its accelerated version Fast-ParC. The ParC operator can capture global features by using a global kernel and circular convolution while keeping location sensitiveness by employing position embeddings. Our Fast-ParC further reduces the O(n2) time complexity of ParC to O(n log n) using Fast Fourier Transform. This acceleration makes it possible to use global convolution in the early stages of models with large feature maps, yet still maintains the overall computational cost comparable with using 3x3 or 7x7 kernels. The proposed operation can be used in a plug-and-play manner to 1) convert ViTs to pure-ConvNet architecture to enjoy wider hardware support and achieve higher inference speed; 2) replacing traditional convolutions in the deep stage of ConvNets to improve accuracy by enlarging the effective receptive field. Experiment results show that our ParC op can effectively enlarge the receptive field of traditional ConvNets, and adopting the proposed op benefits both ViTs and ConvNet models on all three popular vision tasks, image classification, object
Probing the sign of the Hubbard interaction by two-particle quantum walks

We consider two identical bosons propagating on a one-dimensional lattice and address the prob- lem of discriminating whether their mutual on-site interaction is attractive or repulsive. We suggest a probing scheme based on the properties of the corresponding two-particle quantum walks, and show that the sign of the interaction introduces specific and detectable features in the dynamics of quantum correlations, thus permitting to discriminate between the two cases. We also discuss how these features are connected to the band-structure of the Hubbard Hamiltonian, and prove that discrimination may be obtained only when the two walkers are initially prepared in a superposition of localized states.
Dissecting adaptive methods in GANs

Adaptive methods are a crucial component widely used for training generative adversarial networks (GANs). While there has been some work to pinpoint the “marginal value of adaptive methods” in standard tasks, it remains unclear why they are still critical for GAN training. In this paper, we formally study how adaptive methods help train GANs; inspired by the grafting method proposed in Agarwal et al. (2020), we separate the magnitude and direction components of the Adam updates, and graft them to the direction and magnitude of SGDA updates respectively. By considering an update rule with the magnitude of the Adam update and the normalized direction of SGD, we empirically show that the adaptive magnitude of Adam is key for GAN training. This motivates us to have a closer look at the class of normalized stochastic gradient descent ascent (nSGDA) methods in the context of GAN training. We propose a synthetic theoretical framework to compare the performance of nSGDA and SGDA for GAN training with neural networks. We prove that in that setting, GANs trained with nSGDA recover all the modes of the true distribution, whereas the same networks trained with SGDA (and any learning rate configuration) suffer from mode collapse. The critical insight in our analysis is that normalizing the gradients forces the discriminator and generator to be updated at the same pace. We also experimentally show that for several datasets, Adam’s performance can be recovered with nSGDA methods.
Developable Quad Meshes

Fig. 1. We make surfaces developable by imposing a novel local condition on the edges and faces of a quad mesh. That mesh does not have to be aligned with any special curves on the developable surface. For these two examples we take a surface represented by a quad mesh, perform a partition into strips along mesh polylines, and make individual strips developable while boundaries remain fixed. We also show the Gauss image, whose curve components correspond to the developable parts of the surface.
A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning

Visual reinforcement learning (RL), which makes decisions directly from high-dimensional visual inputs, has demonstrated significant potential in various domains. However, deploying visual RL techniques in the real world remains challenging due to their low sample efficiency and large generalization gaps. To tackle these obstacles, data augmentation (DA) has become a widely used technique in visual RL for acquiring sample-efficient and generalizable policies by diversifying the training data. This survey aims to provide a timely and essential review of DA techniques in visual RL in recognition of the thriving development in this field. In particular, we propose a unified framework for analyzing visual RL and understanding the role of DA in it. We then present a principled taxonomy of the existing augmentation techniques used in visual RL and conduct an in-depth discussion on how to better leverage augmented data in different scenarios. Moreover, we report a systematic empirical evaluation of DA-based techniques in visual RL and conclude by highlighting the directions for future research. As the first comprehensive survey of DA in visual RL, this work is expected to offer valuable guidance to this emerging field.
Improved time integration for phase‐field crystal models of solidification

We optimize a numerical time‐stabilization routine for a class of phase‐field crystal (PFC) models of solidification. By numerical experiments, we demonstrate that our simple approach can improve the accuracy of underlying time integration schemes by a few orders of magnitude. We investigate different time integration schemes. Moreover, as a prototypical example for applications, we extend our numerical approach to a PFC model of solidification with an explicit temperature coupling.
Molecular Gas Reservoirs in Massive Quiescent Galaxies at z ∼ 0.7 Linked to Late-time Star Formation

We explore how the presence of detectable molecular gas depends on the inferred star formation histories (SFHs) in eight massive, quiescent galaxies at z ∼ 0.7. Half of the sample have clear detections of molecular gas, traced by CO(2–1). We find that the molecular gas content is unrelated to the rate of star formation decline prior to the most recent 1 Gyr, suggesting that the gas reservoirs are not left over from their primary star formation epoch. However, the recent SFHs of CO-detected galaxies demonstrate evidence for secondary bursts of star formation in their last Gyr. The fraction of stellar mass formed in these secondary bursts ranges from f burst ≈ 0.3%–6% and ended between t end-burst ≈ 0–330 Myr ago. The CO-detected galaxies form a higher fraction of mass in the last Gyr ( fM1Gyr=2.6%±1.8% ) compared to the CO-undetected galaxies ( fM1Gyr=0.2%±0.1% ). The galaxies with gas reservoirs have enhanced late-time star formation, highlighting this as a contributing factor to the observed heterogeneity in the gas reservoirs in high-redshift quiescent galaxies. We find that the amount of gas and star formation driven by these secondary bursts are inconsistent with that expected from dry minor mergers, and instead are likely driven by recently accreted gas, i.e., gas-rich minor mergers. This conclusion would not have been made based on SFRUV+IR measurements alone, highlighting the power of detailed SFH modeling in the interpretation of gas reservoirs. Larger samples are needed to understand the frequency of low-level rejuvenation among quiescent galaxies at intermediate redshifts, and to what extent this drives the diversity of molecular gas reservoirs.
Generalized Dedekind's theorem and its application to integer group determinants

In this paper, we give a refinement of a generalized Dedekind's theorem. In addition, we show that all possible values of integer group determinants of any group are also possible values of integer group determinants of its any abelian subgroup. By applying the refinement of a generalized Dedekind's theorem, we determine all possible values of integer group determinants of the direct product group of the cyclic group of order $8$ and the cyclic group of order $2$.
A Beam-Space Active Sensing Scheme for Integrated Communication and Sensing Applications

In this paper, we develop an active sensing strategy for a millimeter wave (mmWave) band Integrated Sensing and Communication (ISAC) system adopting a realistic hybrid digital-analog (HDA) architecture. To maintain a desired SNR level, initial beam acquisition (BA) must be established prior to data transmission. In the considered setup, a Base Station (BS) Tx transmits data via a digitally modulated waveform and a co-located radar receiver simultaneously performs radar estimation from the backscattered signal. In this BA scheme a single common data stream is broadcast over a wide angular sector such that the radar receiver can detect the presence of not yet acquired users and perform coarse parameter estimation (angle of arrival, time of ﬂight, and Doppler). As a result of the HDA architecture, we consider the design of multi-block adaptive RF-domain “reduction matrices” (from antennas to RF chains) at the radar receiver, to achieve a compromise between the exploration capability in the angular domain and the directivity of the beamforming patterns. Our numerical results demonstrate that the proposed approach is able to reliably detect multiple targets while signiﬁcantly reducing the initial acquisition time.