Miracle-dz/UniFAR-FTUs huggingface dataset:
Contrastive Neighborhood Alignment

We present Contrastive Neighborhood Alignment (CNA), a manifold learning approach to maintain the topology of learned features whereby data points that are mapped to nearby representations by the source (teacher) model are also mapped to neighbors by the target (student) model. The target model aims to mimic the local structure of the source representation space using a contrastive loss. CNA is an unsupervised learning algorithm that does not require ground-truth labels for the individual samples. CNA is illustrated in three scenarios: manifold learning, where the model maintains the local topology of the original data in a dimension-reduced space; model distillation, where a small student model is trained to mimic a larger teacher; and legacy model update, where an older model is replaced by a more powerful one. Experiments show that CNA is able to capture the manifold in a high-dimensional space and improves performance compared to the competing methods in their domains.
Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

The problem of linear predictions has been extensively studied for the past century under pretty generalized frameworks. Recent advances in the robust statistics literature allow us to analyze robust versions of classical linear models through the prism of Median of Means (MoM). Combining these approaches in a piecemeal way might lead to ad-hoc procedures, and the restricted theoretical conclusions that underpin each individual contribution may no longer be valid. To meet these challenges coherently, in this study, we offer a unified robust framework that includes a broad variety of linear prediction problems on a Hilbert space, coupled with a generic class of loss functions. Notably, we do not require any assumptions on the distribution of the outlying data points ($\mathcal{O}$) nor the compactness of the support of the inlying ones ($\mathcal{I}$). Under mild conditions on the dual norm, we show that for misspecification level $\epsilon$, these estimators achieve an error rate of $O(\max\left\{|\mathcal{O}|^{1/2}n^{-1/2}, |\mathcal{I}|^{1/2}n^{-1} \right\}+\epsilon)$, matching the best-known rates in literature. This rate is slightly slower than the classical rates of $O(n^{-1/2})$, indicating that we need to pay a price in terms of error rates to obtain robust estimates. Additionally, we show that this rate can be improved to achieve so-called"fast rates"under additional assumptions.
TransVPR: Transformer-Based Place Recognition with Multi-Level Attention Aggregation

Visual place recognition is a challenging task for applications such as autonomous driving navigation and mobile robot localization. Distracting elements presenting in complex scenes often lead to deviations in the perception of visual place. To address this problem, it is crucial to integrate information from only task-relevant regions into image representations. In this paper, we introduce a novel holistic place recognition model, TransVPR, based on vision Transformers. It benefits from the desirable property of the self-attention operation in Transformers which can naturally aggregate task-relevant features. Attentions from multiple levels of the Transformer, which focus on different regions of interest, are further combined to generate a global image representation. In addition, the output tokens from Transformer layers filtered by the fused attention mask are considered as key-patch descriptors, which are used to perform spatial matching to re-rank the candidates retrieved by the global image features. The whole model allows end-to-end training with a single objective and image-level supervision. TransVPR achieves state-of-the-art performance on several real-world benchmarks while maintaining low computational time and storage requirements.
A Unified Framework for Attention-Based Few-Shot Object Detection

Few-Shot Object Detection (FSOD) is a rapidly growing field in computer vision. It consists in finding all occurrences of a given set of classes with only a few annotated examples for each class. Numerous methods have been proposed to address this challenge and most of them are based on attention mechanisms. However, the great variety of classic object detection frameworks and training strategies makes performance comparison between methods difficult. In particular, for attention-based FSOD methods, it is laborious to compare the impact of the different attention mechanisms on performance. This paper aims at filling this shortcoming. To do so, a flexible framework is proposed to allow the implementation of most of the attention techniques available in the literature. To properly introduce such a framework, a detailed review of the existing FSOD methods is firstly provided. Some different attention mechanisms are then reimplemented within the framework and compared with all other parameters fixed.
SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection

Although point-based networks are demonstrated to be accurate for 3D point cloud modeling, they are still falling behind their voxel-based competitors in 3D detection. We observe that the prevailing set abstraction design for down-sampling points may maintain too much unimportant background information that can affect feature learning for detecting objects. To tackle this issue, we propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA). Technically, we first add a binary segmentation module as the side output to help identify foreground points. Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection. Additionally, it is an easy-to-plug-in module and able to boost various point-based detectors, including single-stage and two-stage ones. Extensive experiments on the popular KITTI and nuScenes datasets validate the superiority of SASA, lifting point-based detection models to reach comparable performance to state-of-the-art voxel-based methods. Code is available at https://github.com/blakechen97/SASA.
Deep Causal Reasoning for Recommendations

Traditional recommender systems aim to estimate a user’s rating to an item based on observed ratings from the population. As with all observational studies, hidden confounders, which are factors that affect both item exposures and user ratings, lead to a systematic bias in the estimation. Consequently, causal inference has been introduced in recommendations to address the influence of unobserved confounders. Observing that confounders in recommendations are usually shared among items and are therefore multi-cause confounders, we model the recommendation as a multi-cause multi-outcome (MCMO) inference problem. Specifically, to remedy the confounding bias, we estimate user-specific latent variables that render the item exposures independent Bernoulli trials. The generative distribution is parameterized by a DNN with factorized logistic likelihood and the intractable posteriors are estimated by variational inference. Controlling these factors as substitute confounders, under mild assumptions, can eliminate the bias incurred by multi-cause confounders. Furthermore, we show that MCMO modeling may lead to high variance due to scarce observations associated with the high-dimensional treatment space. Therefore, we theoretically demonstrate that controlling user features as pre-treatment variables can substantially improve sample efficiency and alleviate overfitting. Empirical studies on both simulated and real-world datasets demonstrate that the proposed deep causal recommender shows more robustness to unobserved confounders than state-of-the-art causal recommenders. Codes and datasets are released at https://github.com/yaochenzhu/Deep-Deconf.
Prediction of ferroelectric superconductors with reversible superconducting diode effect

A noncentrosymmetric superconductor can have a superconducting diode eﬀect, where the critical current in opposite directions is diﬀerent when time-reversal symmetry is also broken. We theoretically propose that a ferroelectric superconductor with coexisting ferroelectricity and superconductivity can support a ferroelectric reversible superconducting diode eﬀect. Through ﬁrst-principles calculation, we predict that monolayer CuNb 2 Se 4 (i.e., bilayer NbSe 2 intercalated with Cu) is such a ferroelectric superconductor, where ferroelectricity controls the layer polarization as well as the sign of spin-orbit coupling induced spin splittings. Because the nonreciprocal eﬀect of the critical current is proportional to the spin splittings, the superconducting diode eﬀect is reversible upon electric switch of ferroelectricity. While we use CuNb 2 Se 4 as a model system, the predicted eﬀect can appear in a class of two-dimensional superconducting bilayers with ferroelectricity induced by interlayer sliding. Our work opens the door to studying the interplay between superconductivity and ferroelectricity in two-dimensional materials.
User-Centric Cooperative Transmissions-Enabled Handover for Ultra-Dense Networks

The user-centric cooperative transmission provides a compelling way to alleviate frequent handovers caused by an ever-increasing number of randomly deployed base stations (BSs) in ultra-dense networks (UDNs). This paper proposes a new user-centric cooperative transmissions-based handover scheme, i.e., the group-cell handover (GCHO) scheme, with the aim of reducing the handover rate in UDNs. In the proposed scheme, the boundary of the cooperating cluster depends on the distance among the user equipment (UE) and cooperating BSs. The new scheme captures the dynamicity and irregularity of the cooperating cluster topology resulting from randomly distributed BSs. Based on stochastic-geometry tools where BSs locations are modeled as the Poison point process (PPP), we derive an analytical expression of the handover rate for the UE with an arbitrary movement trajectory. Furthermore, a GCHO skipping (GCHO-S) scheme is proposed to minimize the handover cost, i.e., the percentage of time wasted in handover signaling in user-centric cooperative transmissions scenarios. The numerical results show that the GCHO scheme decreases the handover rate by 42.3% and 72.7% compared with the traditional single BS association and fixed-region cooperative network topology handover approaches, respectively. Moreover, under the same group-cell size and constant velocity, the GCHO-S scheme diminishes the handover cost by 50% against the GCHO scheme.
Compact Bidirectional Transformer for Image Captioning

Most current image captioning models typically generate captions from left to right. This unidirectional property makes them can only leverage past context but not future context. Though recent refinement-based models can exploit both past and future context by generating a new caption in the second stage based on pre-retrieved or pre-generated captions in the first stage, the decoder of these models generally consists of two networks~(i.e. a retriever or captioner in the first stage and a refiner in the second stage), which can only be executed sequentially. In this paper, we introduce a Compact Bidirectional Transformer model for image captioning that can leverage bidirectional context implicitly and explicitly while the decoder can be executed parallelly. Specifically, it is implemented by tightly coupling left-to-right(L2R) and right-to-left(R2L) flows into a single compact model~(i.e. implicitly) and optionally allowing interaction of the two flows(i.e. explicitly), while the final caption is chosen from either L2R or R2L flow in a sentence-level ensemble manner. We conduct extensive ablation studies on the MSCOCO benchmark and find that the compact architecture, which serves as a regularization for implicitly exploiting bidirectional context, and the sentence-level ensemble play more important roles than the explicit interaction mechanism. By combining with word-level ensemble seamlessly, the effect of the sentence-level ensemble is further enlarged. We further extend the conventional one-flow self-critical training to the two-flows version under this architecture and achieve new state-of-the-art results in comparison with non-vision-language-pretraining models. Source code is available at {\color{magenta}\url{https://github.com/YuanEZhou/CBTrans}}.
Flow-Guided Sparse Transformer for Video Deblurring

Exploiting similar and sharper scene patches in spatio-temporal neighborhoods is critical for video deblurring. However, CNN-based methods show limitations in capturing long-range dependencies and modeling non-local self-similarity. In this paper, we propose a novel framework, Flow-Guided Sparse Transformer (FGST), for video deblurring. In FGST, we customize a self-attention module, Flow-Guided Sparse Window-based Multi-head Self-Attention (FGSW-MSA). For each $query$ element on the blurry reference frame, FGSW-MSA enjoys the guidance of the estimated optical flow to globally sample spatially sparse yet highly related $key$ elements corresponding to the same scene patch in neighboring frames. Besides, we present a Recurrent Embedding (RE) mechanism to transfer information from past frames and strengthen long-range temporal dependencies. Comprehensive experiments demonstrate that our proposed FGST outperforms state-of-the-art (SOTA) methods on both DVD and GOPRO datasets and even yields more visually pleasing results in real video deblurring. Code and pre-trained models are publicly available at https://github.com/linjing7/VR-Baseline
Does entity abstraction help generative Transformers reason?

We study the utility of incorporating entity type abstractions into pre-trained Transformers and test these methods on four NLP tasks requiring different forms of logical reasoning: (1) compositional language understanding with text-based relational reasoning (CLUTRR), (2) abductive reasoning (ProofWriter), (3) multi-hop question answering (HotpotQA), and (4) conversational question answering (CoQA). We propose and empirically explore three ways to add such abstraction: (i) as additional input embeddings, (ii) as a separate sequence to encode, and (iii) as an auxiliary prediction task for the model. Overall, our analysis demonstrates that models with abstract entity knowledge performs better than without it. The best abstraction aware models achieved an overall accuracy of 88.8% and 91.8% compared to the baseline model achieving 62.9% and 89.8% on CLUTRR and ProofWriter respectively. However, for HotpotQA and CoQA, we find that F1 scores improve by only 0.5% on average. Our results suggest that the benefit of explicit abstraction is significant in formally defined logical reasoning settings requiring many reasoning hops, but point to the notion that it is less beneficial for NLP tasks having less formal logical structure.
Online Nonnegative CP-dictionary Learning for Markovian Data

Online Tensor Factorization (OTF) is a fundamental tool in learning low-dimensional interpretable features from streaming multi-modal data. While various algorithmic and theoretical aspects of OTF have been investigated recently, a general convergence guarantee to stationary points of the objective function without any incoherence or sparsity assumptions is still lacking even for the i.i. d. case. In this work, we introduce a novel algorithm that learns a CANDECOMP/PARAFAC (CP) basis from a given stream of tensor-valued data under general constraints, including nonnegativity constraints that induce interpretability of the learned CP basis. We prove that our algorithm converges almost surely to the set of stationary points of the objective function under the hypothesis that the sequence of data tensors is generated by an underlying Markov chain. Our setting covers the classical i.i. d. case as well as a wide range of application contexts including data streams generated by independent or MCMC sampling. Our result closes a gap between OTF and Online Matrix Factorization in global convergence analysis \commHL{for CP-decompositions}. Experimentally, we show that our algorithm converges much faster than standard algorithms for nonnegative tensor factorization tasks on both synthetic and real-world data. Also, we demonstrate the utility of our algorithm on a diverse set of examples from image, video, and time-series data, illustrating how one may learn qualitatively different CP-dictionaries from the same tensor data by exploiting the tensor structure in multiple ways.
Sharp decay rate for the damped wave equation with convex-shaped damping

We revisit the damped wave equation on two-dimensional torus where the damped region does not satisfy the geometric control condition. It was shown in [AL14] that, for sufficiently regular damping, the damped wave equation is stale at a rate sufficiently close to t. We show that if the damping vanishes like a Hölder function |x| , and in addition, the boundary of the damped region is locally strictly convex with positive curvature, the wave is stable at rate t −1+ 2 2β+7 , which is better than the known optimal decay rate t −1+ 1 β+3 for strip-shaped dampings of the same Hölder regularity. Moreover, we show by example that the decay rate is optimal. This illustrates the fact that the sharp energy decay rate depends not only on the order of vanishing of the damping, but also on the shape of the damped region. The main ingredient of the proof is the averaging method (normal form reduction) developed by Hitrik and Sjöstrand ([Hi1][Sj]).
A Critical Study on Data Leakage in Recommender System Offline Evaluation

Recommender models are hard to evaluate, particularly under offline setting. In this article, we provide a comprehensive and critical analysis of the data leakage issue in recommender system offline evaluation. Data leakage is caused by not observing global timeline in evaluating recommenders e.g., train/test data split does not follow global timeline. As a result, a model learns from the user-item interactions that are not expected to be available at the prediction time. We first show the temporal dynamics of user-item interactions along global timeline, then explain why data leakage exists for collaborative filtering models. Through carefully designed experiments, we show that all models indeed recommend future items that are not available at the time point of a test instance, as the result of data leakage. The experiments are conducted with four widely used baseline models—BPR, NeuMF, SASRec, and LightGCN, on four popular offline datasets—MovieLens-25M, Yelp, Amazon-music, and Amazon-electronic, adopting leave-last-one-out data split.1 We further show that data leakage does impact models’ recommendation accuracy. Their relative performance orders thus become unpredictable with different amount of leaked future data in training. To evaluate recommendation systems in a realistic manner in offline setting, we propose a timeline scheme, which calls for a revisit of the recommendation model design.
JABER and SABER: Junior and Senior Arabic BERt

Language-specific pre-trained models have proven to be more accurate than multilingual ones in a monolingual evaluation setting, Arabic is no exception. However, we found that previously released Arabic BERT models were significantly under-trained. In this technical report, we present JABER and SABER, Junior and Senior Arabic BERt respectively, our pre-trained language model prototypes dedicated for Arabic. We conduct an empirical study to systematically evaluate the performance of models across a diverse set of existing Arabic NLU tasks. Experimental results show that JABER and SABER achieve state-of-the-art performances on ALUE, a new benchmark for Arabic Language Understanding Evaluation, as well as on a well-established NER benchmark.
Treating sarcoidosis-associated progressive multifocal leukoencephalopathy with infliximab

Abstract Although most of the progressive multifocal leukoencephalopathy cases in sarcoidosis patients are explained by the treatment with immunosuppressive drugs, it is also reported in treatment-naive sarcoidosis patients, which implies a general predisposition of sarcoidosis patients for progressive multifocal leukoencephalopathy. Indeed, it was shown that active sarcoidosis patients have increased regulatory T cell frequencies which could lead to a subsequent systemic immunosuppression. However, if sarcoidosis with systemic changes of T cell subsets frequencies constitute a risk factor for the development of progressive multifocal leukoencephalopathy, which could then be counteracted by sarcoidosis treatment, is not known. In this cohort study, we included, characterized and followed-up six patients with bioptically confirmed definite progressive multifocal leukoencephalopathy and definite or probable sarcoidosis presenting between April 2013 and January 2019, four of them had no immunosuppressive therapy at the time of developing first progressive multifocal leukoencephalopathy symptoms. Analysis of immune cell subsets in these patients revealed significant imbalances of CD4+ T cell and regulatory T cell frequencies. Due to the progression of progressive multifocal leukoencephalopathy in four patients, we decided to treat sarcoidosis anticipating normalization of immune cell subset frequencies and thereby improving progressive multifocal leukoencephalopathy. Notably, by treatment with infliximab, an antibody directed against tumour necrosis factor-α, three patients continuously improved clinically, JC virus was no longer detectable in the cerebrospinal fluid and regulatory T cell frequencies decreased. One patient was initially misdiagnosed as neurosarcoidosis and died 9 weeks after treatment initiation due to aspiration pneumonia. Our study provides insight that sarcoidosis can lead to changes in T cell subset frequencies, which predisposes to progressive multifocal leukoencephalopathy. Although immunosuppressive drugs should be avoided in progressive multifocal leukoencephalopathy, paradoxically in patients with sarcoidosis treatment with the immunosuppressive infliximab might restore normal T cell distribution and thereby halt progressive multifocal leukoencephalopathy progression.