Miracle-dz/UniFAR-FTUs huggingface dataset:
Vision-and-Language Pretraining

With the burgeoning amount of data of image-text pairs and diversity of Vision-and-Language (V\&L) tasks, scholars have introduced an abundance of deep learning models in this research domain. Furthermore, in recent years, transfer learning has also shown tremendous success in Computer Vision for tasks such as Image Classification, Object Detection, etc., and in Natural Language Processing for Question Answering, Machine Translation, etc. Inheriting the spirit of Transfer Learning, research works in V\&L have devised multiple pretraining techniques on large-scale datasets in order to enhance the performance of downstream tasks. The aim of this article is to provide a comprehensive revision of contemporary V\&L pretraining models. In particular, we categorize and delineate pretraining approaches, along with the summary of state-of-the-art vision-and-language pretrained models. Moreover, a list of training datasets and downstream tasks is supplied to further polish the perspective into V\&L pretraining. Lastly, we decided to take a further step to discuss numerous directions for future research.
Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in VIS and NIR Scenario

In recent years, with the rapid development of face editing and generation, more and more fake videos are circulating on social media, which has caused extreme public concerns. Existing face forgery detection methods based on frequency domain find that the GAN forged images have obvious grid-like visual artifacts in the frequency spectrum. But for synthesized videos, these methods only confine to a single frame and pay little attention to the most discriminative part and temporal frequency clue among different frames. To take full advantage of the rich information in video sequences, this paper performs video forgery detection on both spatial and temporal frequency domains and proposes a Discrete Cosine Transform-based Forgery Clue Augmentation Network (FCAN-DCT) to achieve a more comprehensive spectrum spatial-temporal feature representation. FCAN-DCT totally consists of a backbone network and two branches: Compact Feature Extraction (CFE) module and Frequency Temporal Attention (FTA) module. We conduct thorough experimental assessments on three visible light (VIS) based datasets (i.e.,, FaceForensics++, Celeb-DF (v2), WildDeepfake), and our self-built video forgery dataset DeepfakeNIR, which is the first video forgery dataset on near-infrared (NIR) modality. The experimental results demonstrate the effectiveness and robustness of our method for detecting forgery videos in both VIS and NIR scenarios.DeepfakeNIR and code are available at https://github.com/AEP-WYK/DeepfakeNIR.
FishFormer: Annulus Slicing-based Transformer for Fisheye Rectification with Efficacy Domain Exploration

Numerous significant progress on fisheye image rectification has been achieved through CNN. Nevertheless, constrained by a fixed receptive field, the global distribution and the local symmetry of the distortion have not been fully exploited. To leverage these two characteristics, we introduced Fishformer that processes the fisheye image as a sequence to enhance global and local perception. We tuned the Transformer according to the structural properties of fisheye images. First, the uneven distortion distribution in patches generated by the existing square slicing method confuses the network, resulting in difficult training. Therefore, we propose an annulus slicing method to maintain the con-sistency of the distortion in each patch, thus perceiving the distortion distribution well. Second, we analyze that different distortion parameters have their own efficacy domains. Hence, the perception of the local area is as important as the global, but Transformer has a weakness for local texture perception. Therefore, we propose a novel layer attention mechanism to enhance the local perception and texture transfer. Our network simultaneously implements global perception and focused local perception decided by the different parameters. Extensive experiments demonstrate that our method provides superior performance compared with state-of-the-art methods.
Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimation

In this work, we propose PolarBEV for vision-based uneven BEV representation learning. To adapt to the foreshortening effect of camera imaging, we rasterize the BEV space both angularly and radially, and introduce polar embedding decomposition to model the associations among polar grids. Polar grids are rearranged to an array-like regular representation for efficient processing. Besides, to determine the 2D-to-3D correspondence, we iteratively update the BEV surface based on a hypothetical plane, and adopt height-based feature transformation. PolarBEV keeps real-time inference speed on a single 2080Ti GPU, and outperforms other methods for both BEV semantic segmentation and BEV instance segmentation. Thorough ablations are presented to validate the design. The code will be released at \url{https://github.com/SuperZ-Liu/PolarBEV}.
Entity Linking in Tabular Data Needs the Right Attention

Understanding the semantic meaning of tabular data requires Entity Linking (EL), in order to associate each cell value to a real-world entity in a Knowledge Base (KB). In this work, we focus on end-to-end solutions for EL on tabular data that do not rely on fact lookup in the target KB. Tabular data contains heterogeneous and sparse context, including column headers, cell values and table captions. We experiment with various models to generate a vector representation for each cell value to be linked. Our results show that it is critical to apply an attention mechanism as well as an attention mask, so that the model can only attend to the most relevant context and avoid information dilution. The most relevant context includes: same-row cells, same-column cells, headers and caption. Computational complexity, however, grows quadratically with the size of tabular data for such a complex model. We achieve constant memory usage by introducing a Tabular Entity Linking Lite model (TELL ) that generates vector representation for a cell based only on its value, the table headers and the table caption. TELL achieves 80.8% accuracy on Wikipedia tables, which is only 0.1% lower than the state-of-the-art model with quadratic memory usage.
Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling

Score-based generative models (SGMs) have recently emerged as a promising class of generative models. However, a fundamental limitation is that their inference is very slow due to a need for many (e.g., 2000) iterations of sequential computations. An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We investigate this problem by viewing the diffusion sampling process as a Metropolis adjusted Langevin algorithm, which helps reveal the underlying cause to be ill-conditioned curvature. Under this insight, we propose a model-agnostic preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. Crucially, PDS is proven theoretically to converge to the original target distribution of a SGM, no need for retraining. Extensive experiments on three image datasets with a variety of resolutions and diversity validate that PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality. In particular, PDS can accelerate by up to 29x on more challenging high resolution (1024x1024) image generation.
Merging numerical renormalization group and intermediate representation to compactify two- and three-point correlators

The vanguard of many-body theory is nowadays dealing with the full frequency dynamics of n-point Green's functions for n higher than two. Numerically, these objects easily become a memory bottleneck, even when working with discrete imaginary-time Matsubara frequencies. Here, we use the intermediate representation (IR) to compactify the two-point Green's function and three-point Fermion-Bose vertex directly on the real frequency axis, on the basis of numerical renormalization group (NRG) data. We empirically observe an upper bound of the relative error when comparing the IR reconstructed signal with the original NRG data, and demonstrate that a IR compacification is possible.
Level statistics of real eigenvalues in non-Hermitian systems

Symmetries associated with complex conjugation and Hermitian conjugation, such as time-reversal symmetry and pseudo-Hermiticity, have great impact on eigenvalue spectra of non-Hermitian random matrices. Here, we show that time-reversal symmetry and pseudo-Hermiticity lead to universal level statistics of non-Hermitian random matrices on and around the real axis. From the extensive numerical calculations of large random matrices, we obtain the five universal level-spacing and level-spacing-ratio distributions of real eigenvalues, each of which is unique to the symmetry class. Furthermore, we analyse spacings of real eigenvalues in physical models, such as bosonic many-body systems and free fermionic systems with disorder and dissipation. We clarify that the level spacings in ergodic (metallic) phases are described by the universal distributions of non-Hermitian random matrices in the same symmetry classes, while the level spacings in many-body localized and Anderson localized phases show the Poisson statistics. We also find that the number of real eigenvalues shows distinct scalings in the ergodic and localized phases in these symmetry classes. These results serve as effective tools for detecting quantum chaos, many-body localization, and real-complex transitions in non-Hermitian systems with symmetries.
What Do Graph Convolutional Neural Networks Learn?

Graph neural networks (GNNs) have gained traction over the past few years for their superior performance in numerous machine learning tasks. Graph Convolutional Neural Networks (GCN) are a common variant of GNNs that are known to have high performance in semi-supervised node classification (SSNC), and work well under the assumption of homophily. Recent literature has highlighted that GCNs can achieve strong performance on heterophilous graphs under certain"special conditions". These arguments motivate us to understand why, and how, GCNs learn to perform SSNC. We find a positive correlation between similarity of latent node embeddings of nodes within a class and the performance of a GCN. Our investigation on underlying graph structures of a dataset finds that a GCN's SSNC performance is significantly influenced by the consistency and uniqueness in neighborhood structure of nodes within a class.
On general weighted extropy of ranked set sampling

In the past six years, a considerable attention has been given to the extropy measure proposed by Lad et al. (2015). Weighted Extropy of Ranked Set Sampling was studied and compared with simple random sampling by Qiu et al. (2022). The general weighted extropy and some results related to it are introduced in this paper. We provide general weighted extropy of ranked set sampling. We also studied characterization results, stochastic comparison and monotone properties of general weighted extropy.
Riemann Solvers for Phase Transition in a Compressible Sharp-Interface Method

In this paper, we consider Riemann solvers with phase transition effects based on the Euler-Fourier equation system. One exact and two approximate solutions of the two-phase Riemann problem are obtained by modelling the phase transition process via the theory of classical irreversible thermodynamics. Closure is obtained by appropriate Onsager coefficients for evaporation and condensation. We use the proposed Riemann solvers in a sharp-interface level-set ghost fluid method to couple the individual phases with each other. The Riemann solvers are validated against molecular dynamics data of evaporating Lennard-Jones truncated and shifted fluid. We further study the effects of phase transition on a shock-drop interaction with the novel approximate Riemann solvers.
Strong bulk Dzyaloshinskii-Moriya interaction in composition-uniform centrosymmetric magnetic single layers

: Dzyaloshinskii-Moriya interaction (DMI) is the key ingredient of chiral spintronic phenomena and the emerging technologies based on such phenomena. A nonzero DMI usually occurs at magnetic interfaces or within non-centrosymmetric single crystals. Here, we report the observation of a strong unexpected DMI within a centrosymmetric polycrystalline ferromagnet that has neither a crystal inversion symmetry breaking nor a composition gradient. This DMI is a bulk effect, increases with the thickness of the magnetic layer, and is insensitive to the symmetry of the interfaces or the neighboring materials. We observe a total DMI strength that is a factor of >2 greater than the highest interfacial DMI in the literature. This DMI most likely arises from the strong spin-orbit coupling, strong orbital hybrization, and a “hidden” long-range asymmetry in the material. Our discovery of the strong unconventional bulk DMI in centrosymmetric, composition-uniform magnetic single layers provide fundamental building blocks for the emerging field of spintronics and will stimulate the exploitation of unconventional spin-orbit phenomena in a wide range of materials.
StyleFlow For Content-Fixed Image to Image Translation

Image-to-image (I2I) translation is a challenging topic in computer vision. We divide this problem into three tasks: strongly constrained translation, normally constrained translation, and weakly constrained translation. The constraint here indicates the extent to which the content or semantic information in the original image is preserved. Although previous approaches have achieved good performance in weakly constrained tasks, they failed to fully preserve the content in both strongly and normally constrained tasks, including photo-realism synthesis, style transfer, and colorization, etc. To achieve content-preserving transfer in strongly constrained and normally constrained tasks, we propose StyleFlow, a new I2I translation model that consists of normalizing flows and a novel Style-Aware Normalization (SAN) module. With the invertible network structure, StyleFlow first projects input images into deep feature space in the forward pass, while the backward pass utilizes the SAN module to perform content-fixed feature transformation and then projects back to image space. Our model supports both image-guided translation and multi-modal synthesis. We evaluate our model in several I2I translation benchmarks, and the results show that the proposed model has advantages over previous methods in both strongly constrained and normally constrained tasks.
Image Amodal Completion: A Survey

Existing computer vision systems can compete with humans in understanding the visible parts of objects, but still fall far short of humans when it comes to depicting the invisible parts of partially occluded objects. Image amodal completion aims to equip computers with human-like amodal completion functions to understand an intact object despite it being partially occluded. The main purpose of this survey is to provide an intuitive understanding of the research hotspots, key technologies and future trends in the field of image amodal completion. Firstly, we present a comprehensive review of the latest literature in this emerging field, exploring three key tasks in image amodal completion, including amodal shape completion, amodal appearance completion, and order perception. Then we examine popular datasets related to image amodal completion along with their common data collection methods and evaluation metrics. Finally, we discuss real-world applications and future research directions for image amodal completion, facilitating the reader's understanding of the challenges of existing technologies and upcoming research trends.
Do Not Take It for Granted: Comparing Open-Source Libraries for Software Development Effort Estimation

In the past two decades, several Machine Learning (ML) libraries have become freely available. Many studies have used such libraries to carry out empirical investigations on predictive Software Engineering (SE) tasks. However, the differences stemming from using one library over another have been overlooked, implicitly assuming that using any of these libraries would provide the user with the same or very similar results. This paper aims at raising awareness of the differences incurred when using different ML libraries for software development effort estimation (SEE), one of most widely studied SE prediction tasks. To this end, we investigate 4 deterministic machine learners as provided by 3 of the most popular ML open-source libraries written in different languages (namely, Scikit-Learn, Caret and Weka). We carry out a thorough empirical study comparing the performance of the machine learners on 5 SEE datasets in the two most common SEE scenarios (i.e., out-of-the-box-ml and tuned-ml) as well as an in-depth analysis of the documentation and code of their APIs. The results of our study reveal that the predictions provided by the 3 libraries differ in 95% of the cases on average across a total of 105 cases studied. These differences are significantly large in most cases and yield misestimations of up to approx. 3,000 hours per project. Moreover, our API analysis reveals that these libraries provide the user with different levels of control on the parameters one can manipulate, and a lack of clarity and consistency, overall, which might mislead users. Our findings highlight that the ML library is an important design choice for SEE studies, which can lead to a difference in performance. However, such a difference is under-documented. We conclude by highlighting open-challenges with suggestions for the developers of libraries as well as for the researchers and practitioners using them.
Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing

Big data processing at the production scale presents a highly complex environment for resource optimization (RO), a problem crucial for meeting performance goals and budgetary constraints of analytical users. The RO problem is challenging because it involves a set of decisions (the partition count, placement of parallel instances on machines, and resource allocation to each instance), requires multi-objective optimization (MOO), and is compounded by the scale and complexity of big data systems while having to meet stringent time constraints for scheduling. This paper presents a MaxCompute based integrated system to support multi-objective resource optimization via fine-grained instance-level modeling and optimization. We propose a new architecture that breaks RO into a series of simpler problems, new fine-grained predictive models, and novel optimization methods that exploit these models to make effective instance-level RO decisions well under a second. Evaluation using production workloads shows that our new RO system could reduce 37--72% latency and 43--78% cost at the same time, compared to the current optimizer and scheduler, while running in 0.02-0.23s.
Hot subdwarfs in close binaries observed from space. I. Orbital, atmospheric, and absolute parameters and the nature of their companions

About a third of the hot subdwarfs of spectral type B, which are mostly core-helium burning objects on the extreme horizontal branch, are found in close binaries with cool, low-mass stellar, substellar, or white dwarf companions. They can show light variations due to different phenomena. We used light curves from the Transiting Exoplanet Survey Satellite and the \textit{K2} space mission to look for more sdB binaries. Their light curves can be used to study the hot subdwarf primaries and their companions and get orbital, atmospheric, and absolute parameters for those systems. By classifying the light variations and combining this with the fit of the spectral energy distribution, the distance derived by the parallaxes obtained by \textit{Gaia} and the atmospheric parameters, we could derive the nature of the primary and secondary in 122 (75\%) of the known sdB binaries and 82 newly found reflection effect systems. We derive absolute masses, radii, and luminosities for a total of 39 hot subdwarfs with cool, low-mass companions, as well 29 known and newly found sdBs with white dwarf companions. The mass distribution of hot subdwarfs with cool, low-mass stellar and substellar companions differs from those with white dwarf companions, implying they come from different populations. By comparing the period and minimum companion mass distributions, we find that there are several different populations of hot subdwarfs with white dwarf binaries. We also derive the first orbital period distribution for hot subdwarfs with cool, low-mass stellar or substellar systems selected from light variations instead of radial velocity variations. It shows a period distribution from 1.5 hours to 35 hours compared to the distribution of hot subdwarfs with white dwarfs, which ranges from 1 hour to 30 days. These period distributions can be used to constrain the previous common envelope phase.
On the parabolic Cauchy problem for quantum graphs with vertex noise

We investigate the parabolic Cauchy problem associated with quantum graphs including Lipschitz or polynomial type nonlinearities and additive Gaussian noise perturbed vertex conditions. The vertex conditions are the standard continuity and Kirchhoff assumptions in each vertex. In the case when only Kirchhoff conditions are perturbed, we can prove existence and uniqueness of a mild solution with continuous paths in the standard state space $\mathcal{H}$ of square integrable functions on the edges. We also show that the solution is Markov and Feller. Furthermore, assuming that the vertex values of the normalized eigenfunctions of the self-adjoint operator governing the problem are uniformly bounded, we show that the mild solution has continuous paths in the fractional domain space associated with the Hamiltonian operator, $\mathcal{H}_{\alpha}$ for $\alpha<\frac{1}{4}$. This is the case when the Hamiltonian operator is the standard Laplacian perturbed by a potential. We also show that if noise is present in both type of vertex conditions, then the problem admits a mild solution with continuous paths in the fractional domain space $\mathcal{H}_{\alpha}$ with $\alpha<-\frac{1}{4}$ only. These regularity results are the quantum graph analogues obtained by da Prato and Zabczyk [9] in case of a single interval and classical boundary Dirichlet or Neumann noise.
Lepton Flavor Violation and Lepton Flavor Universality Violation in b and c Decays

Two topics have recently risen to prominence within the ongoing searches of beyond–Standard Model effects in b and c decays: observables that test lepton flavor universality (LFU) and those that test lepton flavor violation (LFV). A coherent set of measurements suggests nonstandard LFU effects. General arguments relate LFU to LFV, and the observed size of the former gives hope of observable signals for the latter. We attempt a comprehensive discussion of both theoretical and experimental aspects of these tests. The main final message is that all the instruments necessary to fully establish the putative new effects are at hand, thanks to running experiments and their upgrades. Therefore, this subject stands a concrete chance of ushering in genuinely unexpected discoveries.