Miracle-dz/UniFAR-FTUs huggingface dataset:
Clique: Spatiotemporal Object Re-identification at the City Scale

Object re-identification (ReID) is a key application of city-scale cameras. While classic ReID tasks are often considered as image retrieval, we treat them as spatiotemporal queries for locations and times in which the target object appeared. Spatiotemporal reID is challenged by the accuracy limitation in computer vision algorithms and the colossal videos from city cameras. We present Clique, a practical ReID engine that builds upon two new techniques: (1) Clique assesses target occurrences by clustering fuzzy object features extracted by ReID algorithms, with each cluster representing the general impression of a distinct object to be matched against the input; (2) to search in videos, Clique samples cameras to maximize the spatiotemporal coverage and incrementally adds cameras for processing on demand. Through evaluation on 25 hours of videos from 25 cameras, Clique reached a high accuracy of 0.87 (recall at 5) across 70 queries and runs at 830x of video realtime in achieving high accuracy.
Some integrable deformations of the Wess-Zumino-Witten model

Lie algebra valued equations translating the integrability of a general two-dimensional Wess-Zumino-Witten model are given. We found simple solutions to these equations and identified three types of new integrable non-linear sigma models. One of them is a modified Yang-Baxter sigma model supplemented with a Wess-Zumino-Witten term.
Kernelized Classification in Deep Networks

In this paper, we propose a kernelized classification layer for deep networks. Although conventional deep networks introduce an abundance of nonlinearity for representation (feature) learning, they almost universally use a linear classifier on the learned feature vectors. We introduce a nonlinear classification layer by using the kernel trick on the softmax cross-entropy loss function during training and the scorer function during testing. Furthermore, we study the choice of kernel functions one could use with this framework and show that the optimal kernel function for a given problem can be learned automatically within the deep network itself using the usual backpropagation and gradient descent methods. To this end, we exploit a classic mathematical result on the positive definite kernels on the unit n-sphere embedded in the (n+1)-dimensional Euclidean space. We show the usefulness of the proposed nonlinear classification layer on several vision datasets and tasks.
Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency

At the heart of many robotics problems is the challenge of learning correspondences across domains. For instance, imitation learning requires obtaining correspondence between humans and robots; sim-to-real requires correspondence between physics simulators and the real world; transfer learning requires correspondences between different robotics environments. This paper aims to learn correspondence across domains differing in representation (vision vs. internal state), physics parameters (mass and friction), and morphology (number of limbs). Importantly, correspondences are learned using unpaired and randomly collected data from the two domains. We propose \textit{dynamics cycles} that align dynamic robot behavior across two domains using a cycle-consistency constraint. Once this correspondence is found, we can directly transfer the policy trained on one domain to the other, without needing any additional fine-tuning on the second domain. We perform experiments across a variety of problem domains, both in simulation and on real robot. Our framework is able to align uncalibrated monocular video of a real robot arm to dynamic state-action trajectories of a simulated arm without paired data. Video demonstrations of our results are available at: this https URL .
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning

We formally study how Ensemble of deep learning models can improve test accuracy, and how the superior performance of ensemble can be distilled into a single model using Knowledge Distillation. We consider the challenging case where the ensemble is simply an average of the outputs of a few independently trained neural networks with the SAME architecture, trained using the SAME algorithm on the SAME data set, and they only differ by the random seeds used in the initialization. We empirically show that ensemble/knowledge distillation in deep learning works very differently from traditional learning theory, especially differently from ensemble of random feature mappings or the neural-tangent-kernel feature mappings, and is potentially out of the scope of existing theorems. Thus, to properly understand ensemble and knowledge distillation in deep learning, we develop a theory showing that when data has a structure we refer to as "multi-view", then ensemble of independently trained neural networks can provably improve test accuracy, and such superior test accuracy can also be provably distilled into a single model by training a single model to match the output of the ensemble instead of the true label. Our result sheds light on how ensemble works in deep learning in a way that is completely different from traditional theorems, and how the "dark knowledge" is hidden in the outputs of the ensemble -- that can be used in knowledge distillation -- comparing to the true data labels. In the end, we prove that self-distillation can also be viewed as implicitly combining ensemble and knowledge distillation to improve test accuracy.
On the Limitations of Denoising Strategies as Adversarial Defenses

As adversarial attacks against machine learning models have raised increasing concerns, many denoising-based defense approaches have been proposed. In this paper, we summarize and analyze the defense strategies in the form of symmetric transformation via data denoising and reconstruction (denoted as $F+$ inverse $F$, $F-IF$ Framework). In particular, we categorize these denoising strategies from three aspects (i.e. denoising in the spatial domain, frequency domain, and latent space, respectively). Typically, defense is performed on the entire adversarial example, both image and perturbation are modified, making it difficult to tell how it defends against the perturbations. To evaluate the robustness of these denoising strategies intuitively, we directly apply them to defend against adversarial noise itself (assuming we have obtained all of it), which saving us from sacrificing benign accuracy. Surprisingly, our experimental results show that even if most of the perturbations in each dimension is eliminated, it is still difficult to obtain satisfactory robustness. Based on the above findings and analyses, we propose the adaptive compression strategy for different frequency bands in the feature domain to improve the robustness. Our experiment results show that the adaptive compression strategies enable the model to better suppress adversarial perturbations, and improve robustness compared with existing denoising strategies.
Symmetric and antisymmetric components of polar-amplified warming

CO2-forced surface warming in general circulation models (GCMs) is initially polar-amplified in the Arctic but not Antarctic—a largely hemispherically antisymmetric signal. Nevertheless, we show in CESM1 and eleven LongRunMIP GCMs that the hemispherically symmetric component of global-mean-normalized, zonal-mean warming () under 4×CO2 changes weakly or becomes modestly more polar-amplified from the first decade to near-equilibrium. Conversely, the antisymmetric warming component () weakens with time in all models, modestly in some including FAMOUS but effectively vanishing in others including CESM1. We explore mechanisms underlying the robust behavior with a diffusive moist energy balance model (MEBM), which given radiative feedback parameter (λ) and ocean heat uptake (𝒪) fields diagnosed from CESM1 adequately reproduces the CESM1 and fields. In further MEBM simulations perturbing λ and 𝒪, is sensitive to their symmetric components only, and more to that of λ. A three-box, two-timescale model fitted to FAMOUS and CESM1 reveals a curiously short Antarctic fast-response timescale in FAMOUS. In additional CESM1 simulations spanning a broader range of forcings, changes modestly across 2-16×CO2, and in a Pliocene-like simulation is more polar-amplified but likewise approximately time-invariant. Determining the real-world relevance of these behaviors—which imply that a surprising amount of information about near-equilibrium polar amplification emerges within decades—merits further study.
The Origin of Strong Linear Polarization from Fast Radio Bursts

The detection of almost 100% linearly polarized emission from the fast radio burst source FRB 121102 implies coherent emission of relativistic electrons moving perpendicular to the ambient magnetic field. The origin of such a particle distribution is very intriguing. Given that FRB 121102 is likely driven by a neutron star, we explored orbits of charged particles trapped in a dipole magnetic field (the Störmer problem). Most previous studies focused on particles with relatively low energies so that the guiding center approximation may be applied. High-energy particles usually have chaotic orbits except those on a periodic orbit or near stable periodic orbits. Via evaluation of the maximum Lyapunov exponent of orbits of particles launched from the equatorial plane with an axial velocity (the angular velocity sets the length and energy scales of the system), we found prominent regions of quasi-periodic orbits around stable periodic orbits in the equatorial plane at high energies. Particles in these orbits oscillate around the equatorial plane and their radial distance from the dipole can vary by a factor of ∼2. Relativistic electrons in such orbits may be responsible for the almost 100% polarized emission from FRB 121102.
Structure of the set of solutions to the Hartree-Fock equation.

The Hartree-Fock equation which is the Euler-Lagrange equation corresponding to the Hartree-Fock energy functional is used in many electron problems. Since the Hartree-Fock equation is a system of nonlinear eigenvalue problems, the study of the structure of the set of all solutions needs new methods different from that for the set of eigenfunctions of linear operators. In this paper we prove that the set of all solutions to the Hartree-Fock equation associated with critical values of the Hartree-Fock energy functional less than the first energy threshold has a structure as a union of real-analytic subsets of a finite number of compact real-analytic manifolds. The result would also be a basis for the study of approximation methods to solve the equation.
Distances to molecular clouds in the second Galactic quadrant

We present distances to 76 medium-sized molecular clouds and an extra large-scale molecular cloud in the second Galactic quadrant (104. °75 < l < 150. °25 and |b| < 5. °25), 73 of which are accurately measured for the first time. Molecular cloud samples are drawn from l-b-V space (− 95 < VLSR < 25 km s−1) with the density-based spatial clustering of applications with noise algorithm, and distances are measured with the background-eliminated extinction-parallax method using extinctions and Gaia DR2 parallaxes. The range of the measured distances to the 76 molecular clouds is from 211 to 2631 pc, and the extra large-scale molecular cloud appears to be a coherent structure at about 1 kpc, across about 40° (~700 pc) in the Galactic longitude.
The Last Journey. II. SMACC—Subhalo Mass-loss Analysis Using Core Catalogs

This paper introduces Subhalo Mass-loss Analysis using Core Catalogs (SMACC). SMACC adds a mass model to substructure merger trees based on halo “core tracking. ” Our approach avoids the need for running expensive subhalo finding algorithms and instead uses subhalo mass-loss modeling to assign masses to halo cores. We present details of the SMACC methodology and demonstrate its excellent performance in describing halo substructure and its evolution. Validation of the approach is carried out using cosmological simulations at significantly different resolutions. We apply SMACC to the 1.24-trillion-particle Last Journey simulation and construct core catalogs with the additional mass information. These catalogs can be readily used as input to semianalytic models or subhalo abundance matching approaches to determine approximate galaxy distributions, as well as for in-depth studies of small-scale structure evolution.
Interactive Speech and Noise Modeling for Speech Enhancement

Speech enhancement is challenging because of the diversity of background noise types. Most of the existing methods are focused on modelling the speech rather than the noise. In this paper, we propose a novel idea to model speech and noise simultaneously in a two-branch convolutional neural network, namely SN-Net. In SN-Net, the two branches predict speech and noise, respectively. Instead of information fusion only at the final output layer, interaction modules are introduced at several intermediate feature domains between the two branches to benefit each other. Such an interaction can leverage features learned from one branch to counteract the undesired part and restore the missing component of the other and thus enhance their discrimination capabilities. We also design a feature extraction module, namely residual-convolution-and-attention (RA), to capture the correlations along temporal and frequency dimensions for both the speech and the noises. Evaluations on public datasets show that the interaction module plays a key role in simultaneous modeling and the SN-Net outperforms the state-of-the-art by a large margin on various evaluation metrics. The proposed SN-Net also shows superior performance for speaker separation.
Semi-Global Shape-aware Network

Non-local operations are usually used to capture long-range dependencies via aggregating global context to each position recently. However, most of the methods cannot preserve object shapes since they only focus on feature similarity but ignore proximity between central and other positions for capturing long-range dependencies, while shape-awareness is beneficial to many computer vision tasks. In this paper, we propose a Semi-Global Shape-aware Network (SGSNet) considering both feature similarity and proximity for preserving object shapes when modeling long-range dependencies. A hierarchical way is taken to aggregate global context. In the first level, each position in the whole feature map only aggregates contextual information in vertical and horizontal directions according to both similarity and proximity. And then the result is input into the second level to do the same operations. By this hierarchical way, each central position gains supports from all other positions, and the combination of similarity and proximity makes each position gain supports mostly from the same semantic object. Moreover, we also propose a linear time algorithm for the aggregation of contextual information, where each of rows and columns in the feature map is treated as a binary tree to reduce similarity computation cost. Experiments on semantic segmentation and image retrieval show that adding SGSNet to existing networks gains solid improvements on both accuracy and efficiency.
Neural Pruning via Growing Regularization

Regularization has long been utilized to learn sparsity in deep neural network pruning. However, its role is mainly explored in the small penalty strength regime. In this work, we extend its application to a new scenario where the regularization grows large gradually to tackle two central problems of pruning: pruning schedule and weight importance scoring. (1) The former topic is newly brought up in this work, which we find critical to the pruning performance while receives little research attention. Specifically, we propose an L2 regularization variant with rising penalty factors and show it can bring significant accuracy gains compared with its one-shot counterpart, even when the same weights are removed. (2) The growing penalty scheme also brings us an approach to exploit the Hessian information for more accurate pruning without knowing their specific values, thus not bothered by the common Hessian approximation problems. Empirically, the proposed algorithms are easy to implement and scalable to large datasets and networks in both structured and unstructured pruning. Their effectiveness is demonstrated with modern deep neural networks on the CIFAR and ImageNet datasets, achieving competitive results compared to many state-of-the-art algorithms. Our code and trained models are publicly available at this https URL.
Orbital Clustering Identifies the Origins of Galactic Stellar Streams

The origins of most stellar streams in the Milky Way are unknown. With improved proper motions provided by Gaia EDR3, we show that the orbits of 23 Galactic stellar streams are highly clustered in orbital phase space. Based on their energies and angular momenta, most streams in our sample can plausibly be associated with a specific (disrupted) dwarf galaxy host that brought them into the Milky Way. For eight streams we also identify likely globular cluster progenitors (four of these associations are reported here for the first time). Some of these stream progenitors are surprisingly far apart, displaced from their tidal debris by a few to tens of degrees. We identify stellar streams that appear spatially distinct, but whose similar orbits indicate they likely originate from the same progenitor. If confirmed as physical discontinuities, they will provide strong constraints on the mass loss from the progenitor. The nearly universal ex situ origin of existing stellar streams makes them valuable tracers of galaxy mergers and dynamical friction within the Galactic halo. Their phase-space clustering can be leveraged to construct a precise global map of dark matter in the Milky Way, while their internal structure may hold clues to the small-scale structure of dark matter in their original host galaxies.
Relevance of longitudinal fields of paraxial optical vortices

Longitudinal electromagnetic fields generally become comparable with the usually dominant transverse components in strongly focused, non-paraxial beams. For paraxial optical vortex modes it is highlighted here how their angular momentum properties produce longitudinal fields that in general must be accounted for. First-order longitudinal components of quantized Laguerre–Gaussian modes are derived and numerically studied with respect to the paraxial parameter, highlighting light-matter and spin-orbit interactions that stem from the longitudinal fields of paraxial beams in free space. New restrictions are cast on the validity of neglecting longitudinal fields for paraxial optical vortices interacting with atoms, molecules and other nanostructures.
An EAGLE view of the missing baryons

Context. A significant fraction of the predicted baryons remain undetected in the local Universe. We adopted the common assumption that a large fraction of the missing baryons correspond to the hot (log T(K) = 5.5–7) phase of the warm-hot intergalactic medium (WHIM). We base our missing baryons search on the scenario whereby the WHIM has been heated up via accretion shocks and galactic outflows, and it is concentrated towards the filaments of the cosmic web. Aims. Our aim is to improve the observational search for the poorly detected hot WHIM. Methods. We detected the filamentary structure within the EAGLE hydrodynamical simulation by applying the Bisous formalism to the galaxy distribution. To test the reliability of our results, we used the MMF/NEXUS+ classification of the large-scale environment of the dark matter component in EAGLE. We then studied the spatio-thermal distribution of the hot baryons within the extracted filaments. Results. While the filaments occupy only ≈5% of the full simulation volume, the diffuse hot intergalactic medium in filaments amounts to ≈23%−25% of the total baryon budget, or ≈79%−87% of all the hot WHIM. The optimal filament sample, with a missing baryon mass fraction of ≈82%, is obtained by selecting Bisous filaments with a high galaxy luminosity density. For these filaments, we derived analytic formulae for the radial gas density and temperature profiles, consistent with recent Planck Sunyaev-Zeldovich and cosmic microwave background lensing observations within the central r ≈ 1 Mpc. Conclusions. Results from the EAGLE simulation suggest that the missing baryons are strongly concentrated towards the filament axes. Since the filament finding methods used here are applicable to galaxy surveys, a large fraction of the missing baryons can be localised by focusing the observational efforts on the central ∼1 Mpc regions of the filaments. To optimise the observational signal, it is beneficial to focus on the filaments with the highest galaxy luminosity densities detected in the optical data.
Dust, Gas, and Metal Content in Star-forming Galaxies at z ∼ 3.3 Revealed with ALMA and Near-IR Spectroscopy

We conducted submillimeter observations with the Atacama Large Millimeter/submillimeter Array (ALMA) of star-forming galaxies at z ∼ 3.3, whose gas-phase metallicities have been measured previously. We investigated the dust and gas contents of the galaxies at z ∼ 3.3 and studied the interaction of galaxies with their circumgalactic or intergalactic medium at this epoch by probing their gas mass fractions and gas-phase metallicities. Single-band dust continuum emission tracing dust mass and the relation between the gas-phase metallicity and gas-to-dust mass ratio were used to estimate the gas masses. The estimated gas mass fractions and depletion timescales are f gas= 0.20–0.75 and t dep= 0.09–1.55 Gyr. Although the galaxies appear to be tightly distributed around the star-forming main sequence at z ∼ 3.3, both quantities show a wider spread at a fixed stellar mass than expected from the scaling relation, suggesting a large diversity of fundamental gas properties in star-forming galaxies that apparently lie on the main sequence. When we compared gas mass fraction and gas-phase metallicity in star-forming galaxies at z ∼ 3.3 and at lower redshifts, star-forming galaxies at z ∼ 3.3 appear to be more metal poor than local galaxies with similar gas mass fractions. Using the gas regulator model to interpret this offset, we find that this can be explained by a higher mass-loading factor, suggesting that the mass-loading factor in outflows increases at earlier cosmic times.
Nonparametric Two-Sample Hypothesis Testing for Random Graphs with Negative and Repeated Eigenvalues.

We propose a nonparametric two-sample test statistic for low-rank, conditionally independent edge random graphs whose edge probability matrices have negative eigenvalues and arbitrarily close eigenvalues. Our proposed test statistic involves using the maximum mean discrepancy applied to suitably rotated rows of a graph embedding, where the rotation is estimated using optimal transport. We show that our test statistic, appropriately scaled, is consistent for sufficiently dense graphs, and we study its convergence under different sparsity regimes. In addition, we provide empirical evidence suggesting that our novel alignment procedure can perform better than the naive alignment in practice, where the naive alignment assumes an eigengap.