Miracle-dz/UniFAR-FTUs huggingface dataset:
$R^{2}$ Former: Unified Retrieval and Reranking Transformer for Place Recognition

Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database. Conventional methods generally adopt aggregated CNN features for global retrieval and RANSAC-based geometric verification for reranking. However, RANSAC only employs geometric information but ignores other possible information that could be useful for reranking, e.g. local feature correlations, and attention values. In this paper, we propose a unified place recognition framework that handles both retrieval and reranking with a novel transformer model, named $R^{2}$ Former. The proposed reranking module takes feature correlation, attention value, and $xy$ coordinates into account, and learns to determine whether the image pair is from the same location. The whole pipeline is end-to-end trainable and the reranking module alone can also be adopted on other CNN or transformer backbones as a generic component. Remarkably, $R^{2}$ Former significantly outperforms state-of-the-art methods on major VPR datasets with much less inference time and memory consumption. It also achieves the state-of-the-art on the hold-out MSLS challenge set and could serve as a simple yet strong solution for real-world large-scale applications. Experiments also show vision transformer tokens are comparable and sometimes better than CNN local features on local matching. The code is released at https://github.com/Jeff-Zilence/R2Former.
ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation

Image keypoints and descriptors play a crucial role in many visual measurement tasks. In recent years, deep neural networks have been widely used to improve the performance of keypoint and descriptor extraction. However, the conventional convolution operations do not provide the geometric invariance required for the descriptor. To address this issue, we propose the sparse deformable descriptor head (SDDH), which learns the deformable positions of supporting features for each keypoint and constructs deformable descriptors. Furthermore, SDDH extracts descriptors at sparse keypoints instead of a dense descriptor map, which enables efficient extraction of descriptors with strong expressiveness. In addition, we relax the neural reprojection error (NRE) loss from dense to sparse to train the extracted sparse descriptors. Experimental results show that the proposed network is both efficient and powerful in various visual measurement tasks, including image matching, 3-D reconstruction, and visual relocalization.
Language-aware Multiple Datasets Detection Pretraining for DETRs

Pretraining on large-scale datasets can boost the performance of object detectors while the annotated datasets for object detection are hard to scale up due to the high labor cost. What we possess are numerous isolated filed-specific datasets, thus, it is appealing to jointly pretrain models across aggregation of datasets to enhance data volume and diversity. In this paper, we propose a strong framework for utilizing Multiple datasets to pretrain DETR-like detectors, termed METR, without the need for manual label spaces integration. It converts the typical multi-classification in object detection into binary classification by introducing a pre-trained language model. Specifically, we design a category extraction module for extracting potential categories involved in an image and assign these categories into different queries by language embeddings. Each query is only responsible for predicting a class-specific object. Besides, to adapt our novel detection paradigm, we propose a group bipartite matching strategy that limits the ground truths to match queries assigned to the same category. Extensive experiments demonstrate that METR achieves extraordinary results on either multi-task joint training or the pretrain&finetune paradigm. Notably, our pre-trained models have high flexible transferability and increase the performance upon various DETR-like detectors on COCO val2017 benchmark. Codes will be available after this paper is published.
Optimal Reads-From Consistency Checking for C11-Style Memory Models

Over the years, several memory models have been proposed to capture the subtle concurrency semantics of C/C++. One of the most fundamental problems associated with a memory model M is consistency checking: given an execution X, is X consistent with M? This problem lies at the heart of numerous applications, including specification testing and litmus tests, stateless model checking, and dynamic analyses. As such, it has been explored extensively and its complexity is well-understood for traditional models like SC and TSO. However, less is known for the numerous model variants of C/C++, for which the problem becomes challenging due to the intricacies of their concurrency primitives. In this work we study the problem of consistency checking for popular variants of the C11 memory model, in particular, the RC20 model, its release-acquire (RA) fragment, the strong and weak variants of RA (SRA and WRA), as well as the Relaxed fragment of RC20. Motivated by applications in testing and model checking, we focus on reads-from consistency checking. The input is an execution X specifying a set of events, their program order and their reads-from relation, and the task is to decide the existence of a modification order on the writes of X that makes X consistent in a memory model. We draw a rich complexity landscape for this problem; our results include (i) nearly-linear-time algorithms for certain variants, which improve over prior results, (ii) fine-grained optimality results, as well as (iii) matching upper and lower bounds (NP-hardness) for other variants. To our knowledge, this is the first work to characterize the complexity of consistency checking for C11 memory models. We have implemented our algorithms inside the TruSt model checker and the C11Tester testing tool. Experiments on standard benchmarks show that our new algorithms improve consistency checking, often by a significant margin.
On the Evaluations of ChatGPT and Emotion-enhanced Prompting for Mental Health Analysis

Automated mental health analysis shows great potential for enhancing the efﬁciency and accessibility of mental health care, whereas the recent dominant methods utilized pre-trained language models (PLMs) as the backbone and incorporated emotional information. The lat-est large language models (LLMs), such as ChatGPT, exhibit dramatic capabilities on diverse natural language processing tasks. However, existing studies on ChatGPT’s zero-shot performance for mental health analysis have limitations in inadequate evaluation, utilization of emotional information, and explainabil-ity of methods. In this work, we comprehensively evaluate the mental health analysis and emotional reasoning ability of ChatGPT on 11 datasets across 5 tasks, including binary and multi-class mental health condition detection, cause/factor detection of mental health conditions, emotion recognition in conversations, and causal emotion entailment. We empirically analyze the impact of different prompting strategies with emotional cues on Chat-GPT’s mental health analysis ability and ex-plainability. Experimental results show that ChatGPT outperforms traditional neural network methods but still has a signiﬁcant gap with advanced task-speciﬁc methods. The qualitative analysis shows its potential in ex-plainability compared with advanced black-box methods but also limitations on robustness and inaccurate reasoning. Prompt engineering with emotional cues is found to be effective in improving its performance on mental health analysis but requires the proper way of emotion infusion.
An expressively complete tangle operator for the topological µ -calculus

The topological µ -calculus has gathered attention in recent years as a powerful framework for representation of spatial knowledge. In particular, spatial relations can be represented over ﬁnite structures in the guise of weakly transitive ( wK4 ) frames. In this paper we show that the topological µ -calculus is equivalent to a simple fragment based on a variant of the ‘tangle’ operator. Similar results were proven for more restricted classes of frames by Anuj Dawar and Martin Otto, using modal characterisation theorems for the corresponding classes of frames. However, since the µ -calculus is not equivalent to the bisimulation invariant fragment of FOL or of MSO over wK4 frames, we use a different method to prove our results; namely, the notion of ﬁnality which was also employed in the proof of completeness and FMP of the µ -calculus over wK4 frames. As a corollary we obtain a new proof of an existing result showing the collapse of the alternation hierarchy over wK4 frames.
Social dynamics through kinetic exchange: the BChS model

This review presents an overview of the current research in kinetic exchange models for opinion formation in a society. The review begins with a brief introduction to previous models and subsequently provides an in-depth discussion of the progress achieved in the Biswas-Chatterjee-Sen model proposed in 2012, also known as the BChS model in some later research publications. The unique feature of the model is its inclusion of negative interaction between agents. The review covers various topics, including phase transitions between different opinion states, critical behavior dependent on various parameters, and applications in realistic scenarios such as the United States presidential election and Brexit.
Graph Enabled Cross-Domain Knowledge Transfer

To leverage machine learning in any decision-making process, one must convert the given knowledge (for example, natural language, unstructured text) into representation vectors that can be understood and processed by machine learning model in their compatible language and data format. The frequently encountered difficulty is, however, the given knowledge is not rich or reliable enough in the first place. In such cases, one seeks to fuse side information from a separate domain to mitigate the gap between good representation learning and the scarce knowledge in the domain of interest. This approach is named Cross-Domain Knowledge Transfer. It is crucial to study the problem because of the commonality of scarce knowledge in many scenarios, from online healthcare platform analyses to financial market risk quantification, leaving an obstacle in front of us benefiting from automated decision making. From the machine learning perspective, the paradigm of semi-supervised learning takes advantage of large amount of data without ground truth and achieves impressive learning performance improvement. It is adopted in this dissertation for cross-domain knowledge transfer. (to be continued)
Polarized images of charged particles in vortical motions around a magnetized Kerr black hole

In this work, we study the images of a Kerr black hole (BH) immersed in uniform magnetic fields, illuminated by the synchrotron radiation of charged particles in the jet. We particularly focus on the spontaneously vortical motions (SVMs) of charged particles in the jet region and investigate the polarized images of electromagnetic radiations from the trajectories along SVMs. We notice that there is a critical value ω_c for charged particle released at a given initial position and subjected an outward force, and once |qB_0/m|=|ω_B|>|ω_c| charged particles can move along SVMs in the jet region. We obtain the polarized images of the electromagnetic radiations from the trajectories along SVMs. Our simplified model suggests that the SVM radiations can act as the light source to illuminate the BH and form a photon ring structure.
Two superconducting states with broken time-reversal symmetry in FeSe1−xSx

Significance When metal becomes superconducting, the Fermi surface becomes unstable, and a superconducting gap forms. In unconventional superconductors, the gap varies in momentum space and can become zero at points or lines on the original Fermi surface. More exotic superconducting states with time-reversal symmetry breaking (TRSB) have been theoretically proposed, but such states are rarely found. Our muon experiments on iron chalcogenide superconductors FeSe1−xSx in the orthorhombic and tetragonal phases provide evidence for TRSB below the transition temperatures. Moreover, the TRSB superconducting state in the tetragonal phase has a unique feature that not all electrons become superconducting. Our results reveal that this system exhibits two distinct superconducting states both with TRSB, which provides research directions for exotic superconductivity.
Distributional Signals for Node Classification in Graph Neural Networks

In graph neural networks (GNNs), both node features and labels are examples of graph signals, a key notion in graph signal processing (GSP). While it is common in GSP to impose signal smoothness constraints in learning and estimation tasks, it is unclear how this can be done for discrete node labels. We bridge this gap by introducing the concept of distributional graph signals. In our framework, we work with the distributions of node labels instead of their values and propose notions of smoothness and non-uniformity of such distributional graph signals. We then propose a general regularization method for GNNs that allows us to encode distributional smoothness and non-uniformity of the model output in semi-supervised node classification tasks. Numerical experiments demonstrate that our method can significantly improve the performance of most base GNN models in different problem settings.
Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field

This work explores the use of 3D generative models to synthesize training data for 3D vision tasks. The key requirements of the generative models are that the generated data should be photorealistic to match the real-world scenarios, and the corresponding 3D attributes should be aligned with given sampling labels. However, we find that the recent NeRF-based 3D GANs hardly meet the above requirements due to their designed generation pipeline and the lack of explicit 3D supervision. In this work, we propose Lift3D, an inverted 2D-to-3D generation framework to achieve the data generation objectives. Lift3D has several merits compared to prior methods: (1) Unlike previous 3D GANs that the output resolution is fixed after training, Lift3D can generalize to any camera intrinsic with higher resolution and photorealistic output. (2) By lifting well-disentangled 2D GAN to 3D object NeRF, Lift3D provides explicit 3D information of generated objects, thus offering accurate 3D annotations for downstream tasks. We evaluate the effectiveness of our framework by augmenting autonomous driving datasets. Experimental results demonstrate that our data generation framework can effectively improve the performance of 3D object detectors. Code: len-li.github. io/lift3d-web
Power deposition studies for standard and crystal-assisted heavy ion collimation in the CERN Large Hadron Collider

The LHC heavy-ion program with $^{208}$Pb$^{82+}$ beams will benefit from a significant increase of the beam intensity when entering its High-Luminosity era in Run~3 (2023). The stored energy is expected to surpass 20~MJ per beam. The LHC is equipped with a betatron collimation system, which intercepts the transverse beam halo and protects sensitive equipment such as superconducting magnets against beam losses. However, nuclear fragmentation and electromagnetic dissociation of $^{208}$Pb$^{82+}$ ions in collimators generates a flux of secondary fragments, which are lost in downstream dispersion suppressor and arc cells. These secondary ions may pose a performance limitation in upcoming runs since they can induce magnet quenches. In order to mitigate this risk, an alternative collimation technique, relying on bent crystals as primary collimators, will be used in forthcoming heavy-ion runs. In this paper, we study the power deposition in superconducting magnets by means of tracking and \textsc{FLUKA} shower simulations, comparing the standard collimation system against the crystal-based one. In order to quantify the predictive ability of the simulation model, we present absolute benchmarks against beam loss monitor measurements from the 2018 $^{208}$Pb$^{82+}$ run at 6.37~$Z$TeV. The benchmarks cover several hundred meters of beamline, from the primary collimators to the first arc cells. Based on these studies, we provide a detailed analysis of ion fragmentation and leakage to the cold magnets and quantify the expected quench margin in future $^{208}$Pb$^{82+}$ runs.
An Analysis of Coincidence between KSE-100 and S&P 500 Index using Spectral Approach

When dealing with time series data, particularly of higher frequency, we are often interested in figuring out periods which are of vital importance. Here in this research, the returns on KSE-100 and S&P 500 index are taken on daily basis from September 2001 to June 2013. As thousands of data points (due to high frequency) are considered, it is impossible for us to figure out any pattern in series, unless suitable filtering is applied on them. For this purpose, a power spectrum will be made by means of a fast fourier transform. This will yield us the events that has influenced KSE-100 index considerably in post 9/11 scenario.
The Dynamics of Leverage and the Belief Distribution of Wealth

The scale and terms of aggregate borrowing in an economy depend on the manner in which wealth is distributed across potential creditors with heterogeneous beliefs about the future. This distribution evolves over time as uncertainty is resolved, in favour of optimists if loans are repaid in full, and in favour of pessimists if there is widespread default. We model this process in an economy with two assets - risky bonds and risk-free cash. Within periods, given the inherited distribution of wealth across belief types, the scale and terms of borrowing are endogenously determined. Following good states, aggregate borrowing and the face value of debt both rise, and the interest rate falls. In the absence of noise, wealth converges to beliefs that differ systematically from the objective probability governing state realisations, with greater risk-aversion associated with greater optimism. In the presence of noise, the economy exhibits periods of high performance, punctuated by periods of crisis and stagnation.
DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium

Self-supervised multi-frame depth estimation achieves high accuracy by computing matching costs of pixel correspondences between adjacent frames, injecting geometric information into the network. These pixel-correspondence candidates are computed based on the relative pose estimates between the frames. Accurate pose predictions are essential for precise matching cost computation as they influence the epipolar geometry. Furthermore, improved depth estimates can, in turn, be used to align pose estimates. Inspired by traditional structure-from-motion (SfM) principles, we propose the DualRefine model, which tightly couples depth and pose estimation through a feedback loop. Our novel update pipeline uses a deep equilibrium model framework to iteratively refine depth estimates and a hidden state of feature maps by computing local matching costs based on epipolar geometry. Importantly, we used the refined depth estimates and feature maps to compute pose updates at each step. This update in the pose estimates slowly alters the epipolar geometry during the refinement process. Experimental results on the KITTI dataset demonstrate competitive depth prediction and odometry prediction performance surpassing published self-supervised baselines11https://github.com/antabangun/DualRefine.
InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning

Recent advances in personalized image generation allow a pre-trained text-to-image model to learn a new concept from a set of images. However, existing personalization approaches usually require heavy test-time finetuning for each concept, which is time-consuming and difficult to scale. We propose InstantBooth, a novel approach built upon pre-trained text-to-image models that enables instant text-guided image personalization without any test-time finetuning. We achieve this with several major components. First, we learn the general concept of the input images by converting them to a textual token with a learnable image encoder. Second, to keep the fine details of the identity, we learn rich visual feature representation by introducing a few adapter layers to the pre-trained model. We train our components only on text-image pairs without using paired images of the same concept. Compared to test-time finetuning-based methods like DreamBooth and Textual-Inversion, our model can generate competitive results on unseen concepts concerning language-image alignment, image fidelity, and identity preservation while being 100 times faster.
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review

The field of deep learning has witnessed significant progress, particularly in computer vision (CV), natural language processing (NLP), and speech. The use of large-scale models trained on vast amounts of data holds immense promise for practical applications, enhancing industrial productivity and facilitating social development. With the increasing demands on computational capacity, though numerous studies have explored the efficient training, a comprehensive summarization on acceleration techniques of training deep learning models is still much anticipated. In this survey, we present a detailed review for training acceleration. We consider the fundamental update formulation and split its basic components into five main perspectives: (1) data-centric: including dataset regularization, data sampling, and data-centric curriculum learning techniques, which can significantly reduce the computational complexity of the data samples; (2) model-centric, including acceleration of basic modules, compression training, model initialization and model-centric curriculum learning techniques, which focus on accelerating the training via reducing the calculations on parameters; (3) optimization-centric, including the selection of learning rate, the employment of large batchsize, the designs of efficient objectives, and model average techniques, which pay attention to the training policy and improving the generality for the large-scale models; (4) budgeted training, including some distinctive acceleration methods on source-constrained situations; (5) system-centric, including some efficient open-source distributed libraries/systems which provide adequate hardware support for the implementation of acceleration algorithms. By presenting this comprehensive taxonomy, our survey presents a comprehensive review to understand the general mechanisms within each component and their joint interaction.
Breaking the Envy Cycle: Best-of-Both-Worlds Guarantees for Subadditive Valuations

We study best-of-both-worlds guarantees for the fair division of indivisible items among agents with subadditive valuations. Our main result establishes the existence of a random allocation that is simultaneously ex-ante $\frac{1}{2}$-envy-free, ex-post $\frac{1}{2}$-EFX and ex-post EF1, for every instance with subadditive valuations. We achieve this result by a novel polynomial-time algorithm that randomizes the well-established envy cycles procedure in a way that provides ex-ante fairness. Notably, this is the first best-of-both-worlds fairness guarantee for subadditive valuations, even when considering only EF1 without EFX.