ShantanuT01/DACTYL-Pretraining huggingface dataset:
The pretrain-finetune paradigm is a classical pipeline in visual learning. Recent progress on unsupervised pretraining methods shows superior transfer performance to their supervised counterparts. This paper revisits this phenomenon and sheds new light on understanding the transferability gap between unsupervised and supervised pretraining from a multilayer perceptron (MLP) perspective. While previous works focus on the effectiveness of MLP on unsupervised image classification where pretraining and evaluation are conducted on the same dataset, we reveal that the MLP projector is also the key factor to better transferability of unsupervised pretraining methods than supervised pretraining methods. Based on this observation, we attempt to close the transferability gap between supervised and unsupervised pretraining by adding an MLP projector before the classifier in supervised pretraining. Our analysis indicates that the MLP projector can help retain intra-class variation of visual features, decrease the feature distribution distance between pretraining and evaluation datasets, and reduce feature redundancy. Extensive experiments on public benchmarks demonstrate that the added MLP projector significantly boosts the transferability of supervised pretraining, e.g. +7.2% top-1 accuracy on the concept generalization task, +5.8% top-1 accuracy for linear evaluation on 12-domain classification tasks, and +0.8% AP on COCO object detection task, making supervised pretraining comparable or even better than unsupervised pretraining.
At the dawn of a new decade, particle physics faces the challenge of explaining the mystery of dark matter, the origin of matter over antimatter in the Universe, the apparent fine-tuning of the electro-weak scale, and many other aspects of fundamental physics. Perhaps the most striking frontier to emerge in the search for answers involves new physics at mass scales comparable to familiar matter, below the GeV scale, but with very feeble interaction strength. New theoretical ideas to address dark matter and other fundamental questions predict such feebly interacting particles (FIPs) at these scales, and indeed, existing data may even provide hints of this possibility. Emboldened by the lessons of the LHC, a vibrant experimental program to discover such physics is under way, guided by a systematic theoretical approach firmly grounded on the underlying principles of the Standard Model. We give an overview of these efforts, their motivations, and the decadal goals that animate the community involved in the search for FIPs, with special focus on accelerator-based experiments.
Deep Neural Networks (DNNs) generalize well despite their massive size and capability of memorizing all examples. There is a hypothesis that DNNs start learning from simple patterns and the hypothesis is based on the existence of examples that are consistently well-classified at the early training stage (i.e., easy examples) and examples misclassified (i.e., hard examples). Easy examples are the evidence that DNNs start learning from specific patterns and there is a consistent learning process. It is important to know how DNNs learn patterns and obtain generalization ability, however, properties of easy and hard examples are not thoroughly investigated (e.g., contributions to generalization and visual appearances). In this work, we study the similarities of easy and hard examples respectively for different Convolutional Neural Network (CNN) architectures, assessing how those examples contribute to generalization. Our results show that easy examples are visually similar to each other and hard examples are visually diverse, and both examples are largely shared across different CNN architectures. Moreover, while hard examples tend to contribute more to generalization than easy examples, removing a large number of easy examples leads to poor generalization. By analyzing those results, we hypothesize that biases in a dataset and Stochastic Gradient Descent (SGD) are the reasons why CNNs have consistent easy and hard examples. Furthermore, we show that large scale classification datasets can be efficiently compressed by using easiness proposed in this work.
This chapter approaches the image retrieval system on the base of the colors of image. It creates fuzzy signature to describe the color of image on color space HSV and builds fuzzy Hamming distance (FHD) to evaluate the similarity between the images. In order to reduce the storage space and speed up the search of similar images, it aims to create S-tree to store fuzzy signature relies on FHD and builds image retrieval algorithm on S-tree. Then, it provides the content-based image retrieval (CBIR) and an image retrieval method on FHD and S-tree. Last but not least, based on this theory, it also presents an application and experimental assessment of the process of querying similar image on the database system over 10,000 images.
This paper is concerned with an inverse obstacle problem which employs the dynamical scattering data of acoustic wave over a finite time interval. The unknown obstacle is assumed to be sound-soft one. The governing equation of the wave is given by the classical wave equation. The wave is generated by the initial data localized outside the obstacle and observed over a finite time interval at a place which is not necessary the same as the support of the initial data. The observed data are the so-called bistatic data. In this paper, an enclosure method which employs the bistatic data and is based on two main analytical formulae, is developed. The first one enables us to extract the maximum spheroid with focal points at the center of the support of the initial data and that of the observation points whose exterior encloses the unknown obstacle of general shape. The second one, under some technical assumption for the obstacle including convexity as an example, indicates the deviation of the geometry of the boundary of the obstacle and the maximum spheroid at the contact points. Several implications of those two formulae are also given. In particular, a constructive proof of a uniqueness of a spherical obstacle using the bistatic data is given.
The nodal-line semimetals have attracted immense interest due to the unique electronic structures such as the linear dispersion and the vanishing density of states as the Fermi energy approaching the nodes. Here, we report temperature-dependent transport and scanning tunneling microscope (spectroscopy) (STM[S]) measurements on nodal-line semimetal ZrSiSe.Our experimental results and theoretical analyses consistently demonstrate that the temperature induces Lifshitz transitions at 80 and 106 K in ZrSiSe, which results in the transport anomalies at the same temperatures. More strikingly, we observe a V-shaped dip structure around Fermi energy from the STS spectrum at low temperature,which can be attributed to co-effect of the spin-orbit coupling and excitonic instability. Our observations indicate the correlation interaction may play an important role in ZrSiSe, which owns the quasi-two-dimensional electronic structures.
We have developed a self-consistent model of the disk around the nearby 10 Myr old star TW Hya which matches the observed spectral energy distribution and 7mm images of the disk. The model requires a significant dust size evolution and a partially-evacuated inner disk region, as predicted by theories of planet formation. The outer disk, which extends to at least 140 AU in radius, is very optically thick at IR wavelengths and quite massive ~0.06 Msun for the relatively advanced age of this T Tauri star. This implies long viscous and dust evolution timescales, although dust must have grown to sizes of order ~1cm to explain the sub-mm and mm spectral slopes. In contrast, the negligible near-infrared excess emission of this system requires that the disk be optically thin inside ~4 AU.This inner region cannot be completely evacuated; we need ~0.5 lunar mass of ~1 micron particles remaining to produce the observed 10 micron silicate emission. Our model requires a distinct transition in disk properties at ~4 AU, separating the inner and outer disk. The inner edge of the optically-thick outer disk must be heated almost frontally by the star to account for the excess flux at mid-IR wavelengths. We speculate that this truncation of the outer disk may be the signpost of a developing gap due to the effects of a growing protoplanet; the gap is still presumably evolving because material still resides in it, as indicated by the silicate emission, the molecular hydrogen emission, and by the continued accretion onto the central star (albeit at a much lower rate than typical of younger T Tauri stars). TW Hya thus may become the Rosetta stone for our understanding of the evolution and dissipation of protoplanetary disks.
Red supergiant stars represent a late stage of the evolution of stars more massive than about nine solar masses, in which they develop complex, multi-component atmospheres. Bright spots have been detected in the atmosphere of red supergiants using interferometric imaging. Above the photosphere of a red supergiant, the molecular outer atmosphere extends up to about two stellar radii. Furthermore, the hot chromosphere (5,000 to 8,000 kelvin) and cool gas (less than 3,500 kelvin) of a red supergiant coexist at about three stellar radii. The dynamics of such complex atmospheres has been probed by ultraviolet and optical spectroscopy. The most direct approach, however, is to measure the velocity of gas at each position over the image of stars as in observations of the Sun. Here we report the mapping of the velocity field over the surface and atmosphere of the nearby red supergiant Antares. The two-dimensional velocity field map obtained from our near-infrared spectro-interferometric imaging reveals vigorous upwelling and downdrafting motions of several huge gas clumps at velocities ranging from about -20 to +20 kilometres per second in the atmosphere, which extends out to about 1.7 stellar radii. Convection alone cannot explain the observed turbulent motions and atmospheric extension, suggesting that an unidentified process is operating in the extended atmosphere.
We consider the problem of the motion of $N$ bodies in a self-gravitating system in two spacetime dimensions. We point out that this system can be mapped onto the quantum-mechanical problem of an N-body generalization of the problem of the H$_{2}^{+}$ molecular ion in one dimension. The canonical gravitational N-body formalism can be extended to include electromagnetic charges. We derive a general algorithm for solving this problem, and show how it reduces to known results for the 2-body and 3-body systems.
In the wake of successful experiments in Fermi condensates, experimental attention is broadening to study resonant interactions in degenerate Bose-Fermi mixtures. Here we consider the properties and stability of the fermionic molecules that can be created in such a mixture near a Feshbach resonance (FR). To do this, we consider the two-body scattering matrix in the many-body environment, and assess its complex poles. The stability properties of these molecules strongly depend on their centre-of-mass motion, because they must satisfy Fermi statistics. At low centre-of-mass momenta the molecules are more stable than in the absence of the environment (due to Pauli-blocking effects), while at high centre-of-mass momenta nontrivial many body effects render them somewhat less stable.
In this study, we present a rutile-related material, LiMoO2, that becomes a cluster magnet and exhibits a spin singlet formation on a preformed molybdenum dimer upon cooling. Unlike ordinary cluster magnets, the atomic dyz orbital robustly survives despite the formation of molecular orbitals, thereby affecting the magnetic properties of the selected material. Such hybrid cluster magnets with the characters of molecular and atomic orbitals realize multiple independent spins on an isolated cluster, leading to an ideal platform to study the isolated spin dimers physics.
In this work, we investigate the dynamical state of the galaxy cluster Abell 2631, a massive structure located at the core of the Saraswati supercluster. To do this, we first solve a tension found in the literature regarding the weak-lensing mass determination of the cluster. We do this through a comprehensive weak-lensing analysis, exploring the power of the combination of shear and magnification data sets. We find $M_{200}^{\rm wl} = 8.7_{-2.9}^{+2.5} \times 10^{14}$ M$_\odot$. We also determined the mass based on the dynamics of spectroscopic members, corresponding to $M_{200}^{\rm dy} = 12.2\pm3.0 \times 10^{14}$ M$_\odot$, consistent within a 68 per cent CL with the weak-lensing estimate. The scenarios provided by the mass distribution and dynamics of galaxies are reconciled with those provided by X-ray observations in a scenario where A2631 is observed at a late stage of merging.
We consider the effects of spacetime curvature and brane thickness on the localization of gauge fields on a brane via kinetic terms induced by localized fermions. We find that in a warped geometry with and infinitely thin brane, both the infrared and the ultraviolet behavior of the electromagnetic propagator are affected, providing a more stringent bound on the brane's tension than that coming from the requirement of four-dimensional gravity on the brane. On the other hand, for a thick wall in a flat spacetime, where the fermions are localized by means of a Yukawa coupling, we find that 4-dimensional electromagnetism is recovered in a region bounded from above by the same critical distance appearing in the thin case, but also from below by a new scale related to the brane's thickness and the electromagnetic couplings.
Since its launch on 22 September 2006, the EUV Imaging Spectrometer onboard the Hinode satellite has exhibited a gradual decay in sensitivity. Using spectroheliograms taken in the Fe VIII 185.21 Angstrom and Si VII 275.35 Angstrom emission lines in quiet regions near Sun center we characterize that decay. For the period from December 2006 to March 2012, the decline in the sensitivity can be characterized as an exponential decay with an average time constant of 7358 +/- 1030 days (20.2 +/- 2.8 years). Emission lines formed at temperatures >10^6.1 K in the quiet-Sun data exhibit solar-cycle effects.
The dielectric response of materials underpins electronics and photonics. Established semiconductor materials have a narrow range of dielectric susceptibility, with low-frequency values on the order of 10. Strong and variable dielectric response in wide-band gap materials is associated with complex crystal structures and heavier elements. Based on underlying chemical trends, we hypothesize that chalcogenides in crystal structures common to complex oxides may feature many highly-polarizable semiconductors. Research on these materials is motivated by fundamental inquiry into electrons and phonons in solids, and by potential applications in photonics, high-frequency communications, and photovoltaics.
Recently, quadrupole helix chirality and its domain structure was observed in resonant x-ray diffraction experiments on DyFe$_3$(BO$_3$)$_4$ using circularly polarized x rays [T. Usui et al., Nature Materials 13, 611 (2014)]. We show that this effect can be explained quantitatively by calculating the quadrupole moments of the Dy$^{3+}$ ions induced by the low-symmetry ($C_2$) crystal-field (CF) component. In this work, the CF parameters for the Dy$^{3+}$ ions in the $P3_121$ ($P3_221$) phase of DyFe$_3$(BO$_3$)$_4$ are obtained from CF calculations based on the analysis of high-resolution temperature-dependent optical spectroscopy data. We also consider the helix chirality of the single-site magnetic susceptibility tensors of the Dy$^{3+}$ ions in the paramagnetic $P3_121$ ($P3_221$) phase and suggest a neutron diffraction experiment to reveal it.
This paper presents a study of finite element error estimation of advection-diffusion-reaction equation with spatially variable coefficients. We have derived a priori and a posteriori errors in both energy and L2 norm. We have used residual-based a posteriori error estimator. Numerical results are also presented to verify our theoretical approach.
I state and prove, in the context of a space having only the metrical and affine structure imposed by the geometrized version of Newtonian gravitational theory, a theorem analagous to that of Weyl's for a Lorentz manifold. The theorem says that a projective structure and a suitably defined compatible conformal structure jointly suffice for fixing the metric structure of a Newtonian spacetime model up to constant factors, and for fixing its affine structure as well. The theorem allows one to give a natural, physically compelling interpretation of the spatiotemporal geometry of a geometrized Newtonian gravity spacetime manifold, in close analogy with the way Weyl's Theorem allows one to do in general relativity.
Analyzing data from GLAST's Large Area Telescope (LAT) will require sophisticated techniques. The PSF and effective area are functions of both photon energy and the position in the field-of-view. During most of the mission the observatory will survey the sky continuously and thus the LAT will detect each count from a source at a different detector orientation; each count requires its own response function! The likelihood as a function of celestial position and photon energy will be the foundation of the standard analysis techniques. However the 20 MeV-300 GeV emission at the time of the ~100 keV burst emission (timescale of ~10 s) can be isolated and analyzed because essentially no non-burst counts are expected within a PSF radius of the burst location during the burst. Both binned and unbinned (in energy) spectral fitting will be possible. Longer timescale afterglow emission will require the likelihood analysis that will be used for persistent sources.
We apply a two-particle semi-analytic approach to a single Anderson impurity attached to two biased metallic leads. The theory is based on reduced parquet equations justified in critical regions of singularities in the Bethe-Salpeter equations. It offers a way to treat one-particle and two-particle thermodynamic and spectral quantities on the same footing. The two-particle vertices are appropriately renormalized so that spurious transitions into the magnetic state of the weak-coupling approximations are suppressed. The unphysical hysteresis loop in the current-voltage characteristics is thereby eliminated. Furthermore, in the linear response regime, we qualitatively reproduce the three transport regimes with the increasing temperature: from the Kondo resonant tunneling through the Coulomb blockade regime up to a sequential tunneling regime. Far from equilibrium, we find that the bias plays a similar role as the temperature in destroying the Kondo resonant peak when the corresponding energy scale is comparable with the Kondo temperature. Besides that, the applied voltage in low bias is shown to develop spectral peaks around the lead chemical potentials as observed in previous theoretical and experimental studies.
We are interested in modeling some two-level population dynamics, resulting from the interplay of ecological interactions and phenotypic variation of individuals (or hosts) and the evolution of cells (or parasites) of two types living in these individuals. The ecological parameters of the individual dynamics depend on the number of cells of each type contained by the individual and the cell dynamics depends on the trait of the invaded individual. Our models are rooted in the microscopic description of a random (discrete) population of individuals characterized by one or several adaptive traits and cells characterized by their type. The population is modeled as a stochastic point process whose generator captures the probabilistic dynamics over continuous time of birth, mutation and death for individuals and birth and death for cells. The interaction between individuals (resp. between cells) is described by a competition between individual traits (resp. between cell types). We look for tractable large population approximations. By combining various scalings on population size, birth and death rates and mutation step, the single microscopic model is shown to lead to contrasting nonlinear macroscopic limits of different nature: deterministic approximations, in the form of ordinary, integro- or partial differential equations, or probabilistic ones, like stochastic partial differential equations or superprocesses. The study of the long time behavior of these processes seems very hard and we only develop some simple cases enlightening the difficulties involved.
Few-shot segmentation (FSS) aims to segment unseen classes given only a few annotated samples. Existing methods suffer the problem of feature undermining, i.e. potential novel classes are treated as background during training phase. Our method aims to alleviate this problem and enhance the feature embedding on latent novel classes. In our work, we propose a novel joint-training framework. Based on conventional episodic training on support-query pairs, we add an additional mining branch that exploits latent novel classes via transferable sub-clusters, and a new rectification technique on both background and foreground categories to enforce more stable prototypes. Over and above that, our transferable sub-cluster has the ability to leverage extra unlabeled data for further feature enhancement. Extensive experiments on two FSS benchmarks demonstrate that our method outperforms previous state-of-the-art by a large margin of 3.7% mIOU on PASCAL-5i and 7.0% mIOU on COCO-20i at the cost of 74% fewer parameters and 2.5x faster inference speed. The source code is available at https://github.com/LiheYoung/MiningFSS.
Gamow-Teller(GT) transitions in the $\beta$ decays of $^9$Li and $^{11}$Li are investigated with theoretical calculations of antisymmetrized molecular dynamics. The calculated $B(GT)$ values are small for transitions to low-lying states in Be isotopes, while relatively large $B(GT)$ values are found for excited states at excitation energy $E_x\ge 10$ MeV. Sum of the $B(GT)$ values are discussed for each spin parity of final states. The calculated results seem to be inconsistent with the experimental report of the strongest GT-transition from $^9$Li to the $5/2^-$ state at 11.8 MeV of $^9$Be.
A simple circuit implementation of the oracle for Grover's quantum search of a real unstructured classical database is proposed. The oracle contains a kind of quantumly accessible classical memory, which stores the database.
The Interacting Boson plus broken-pairs Model has been used to describe high-spin bands in spherical and transitional nuclei. In the spherical nucleus $^{104}$Cd the model reproduces the structure of high-spin bands built on a vibrational structure. Model calculations performed with a single set of parameters reproduce ten bands in $^{136}$Nd, including the two dipole high-spin bands based on the $(\pi h_{11/2})^2$ $(\nu h_{11/2})^2$ configuration.