Miracle-dz/UniFAR-FTUs huggingface dataset:
Extrapolation and Factorization of matrix weights

In this paper we prove the Jones factorization theorem and the Rubio de Francia extrapolation theorem for matrix $\mathcal A_p$ weights. These results answer longstanding open questions in the study of matrix weights. The proof requires the development of the theory of convex-set valued functions and measurable seminorm functions. In particular, we define a convex-set valued version of the Hardy Littlewood maximal operator and construct an appropriate generalization of the Rubio de Francia iteration algorithm, which is central to the proof of both results in the scalar case.
Wijesekera-style constructive modal logics

We define a family of propositional constructive modal logics corresponding each to a different classical modal system. The logics are defined in the style of Wijesekera's constructive modal logic, and are both proof-theoretically and semantically motivated. On the one hand, they correspond to the single-succedent restriction of standard sequent calculi for classical modal logics. On the other hand, they are obtained by incorporating the hereditariness of intuitionistic Kripke models into the classical satisfaction clauses for modal formulas. We show that, for the considered classical logics, the proof-theoretical and the semantical approach return the same constructive systems.
Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel

Identifying unfamiliar inputs, also known as out-of-distribution (OOD) detection, is a crucial property of any decision making process. A simple and empirically validated technique is based on deep ensembles where the variance of predictions over different neural networks acts as a substitute for input uncertainty. Nevertheless, a theoretical understanding of the inductive biases leading to the performance of deep ensemble's uncertainty estimation is missing. To improve our description of their behavior, we study deep ensembles with large layer widths operating in simplified linear training regimes, in which the functions trained with gradient descent can be described by the neural tangent kernel. We identify two sources of noise, each inducing a distinct inductive bias in the predictive variance at initialization. We further show theoretically and empirically that both noise sources affect the predictive variance of non-linear deep ensembles in toy models and realistic settings after training. Finally, we propose practical ways to eliminate part of these noise sources leading to significant changes and improved OOD detection in trained deep ensembles.
Transfer-learning for video classification: Video Swin Transformer on multiple domains

The computer vision community has seen a shift from convolutional-based to pure transformer architectures for both image and video tasks. Training a transformer from zero for these tasks usually requires a lot of data and computational resources. Video Swin Transformer (VST) is a pure-transformer model developed for video classification which achieves state-of-the-art results in accuracy and efficiency on several datasets. In this paper, we aim to understand if VST generalizes well enough to be used in an out-of-domain setting. We study the performance of VST on two large-scale datasets, namely FCVID and Something-Something using a transfer learning approach from Kinetics-400, which requires around 4x less memory than training from scratch. We then break down the results to understand where VST fails the most and in which scenarios the transfer-learning approach is viable. Our experiments show an 85\% top-1 accuracy on FCVID without retraining the whole model which is equal to the state-of-the-art for the dataset and a 21\% accuracy on Something-Something. The experiments also suggest that the performance of the VST decreases on average when the video duration increases which seems to be a consequence of a design choice of the model. From the results, we conclude that VST generalizes well enough to classify out-of-domain videos without retraining when the target classes are from the same type as the classes used to train the model. We observed this effect when we performed transfer-learning from Kinetics-400 to FCVID, where most datasets target mostly objects. On the other hand, if the classes are not from the same type, then the accuracy after the transfer-learning approach is expected to be poor. We observed this effect when we performed transfer-learning from Kinetics-400, where the classes represent mostly objects, to Something-Something, where the classes represent mostly actions.
Probing Cross-modal Semantics Alignment Capability from the Textual Perspective

In recent years, vision and language pre-training (VLP) models have advanced the state-of-the-art results in a variety of cross-modal downstream tasks. Aligning cross-modal semantics is claimed to be one of the essential capabilities of VLP models. However, it still remains unclear about the inner working mechanism of alignment in VLP models. In this paper, we propose a new probing method that is based on image captioning to first empirically study the cross-modal semantics alignment of VLP models. Our probing method is built upon the fact that given an image-caption pair, the VLP models will give a score, indicating how well two modalities are aligned; maximizing such scores will generate sentences that VLP models believe are of good alignment. Analyzing these sentences thus will reveal in what way different modalities are aligned and how well these alignments are in VLP models. We apply our probing method to five popular VLP models, including UNITER, ROSITA, ViLBERT, CLIP, and LXMERT, and provide a comprehensive analysis of the generated captions guided by these models. Our results show that VLP models (1) focus more on just aligning objects with visual words, while neglecting global semantics; (2) prefer fixed sentence patterns, thus ignoring more important textual information including fluency and grammar; and (3) deem the captions with more visual words are better aligned with images. These findings indicate that VLP models still have weaknesses in cross-modal semantics alignment and we hope this work will draw researchers' attention to such problems when designing a new VLP model.
A Systematic Study of the Consistency of Two-Factor Authentication User Journeys on Top-Ranked Websites (Extended Version)

Heuristics for user experience state that users will transfer their expectations from one product to another. A lack of consistency between products can increase users' cognitive friction, leading to frustration and rejection. This paper presents the first systematic study of the external, functional consistency of two-factor authentication user journeys on top-ranked websites. We find that these websites implement only a minimal number of design aspects consistently (e.g., naming and location of settings) but exhibit mixed design patterns for setup and usage of a second factor. Moreover, we find that some of the more consistently realized aspects, such as descriptions of two-factor authentication, have been described in the literature as problematic and adverse to user experience. Our results advocate for more general UX guidelines for 2FA implementers and raise new research questions about the 2FA user journeys.
Constraining the Origin of Stellar Binary Black Hole Mergers by Detections of Their Lensed Host Galaxies and Gravitational Wave Signals

A significant number of stellar binary black hole (sBBH) mergers may be lensed and detected by the third generation of gravitational wave (GW) detectors. Their lensed host galaxies may be detectable, which would thus help to accurately localize these sources and provide a new approach to study the origin of sBBHs. In this paper, we investigate the detectability of lensed host galaxies for lensed sBBH mergers. We find that the detection fraction of galaxies hosting lensed GW events can be significantly different for a survey with a given limiting magnitude if sBBHs are produced by different mechanisms, such as the evolution of massive binary stars, dynamical interactions in dense star clusters, and production assisted by active galactic nuclei or massive black holes. Furthermore, we illustrate that the statistical spatial distributions of those lensed sBBHs in their hosts resulting from different sBBH formation channels can differ. Therefore, with the third generation of GW detectors and future large-scale galaxy surveys, it is possible to independently constrain the origin of sBBHs via the detection fraction of those lensed events with identifiable lensing host signatures and/or even to constrain the fractional contributions from different sBBH formation mechanisms.
On the number of even values of an eta-quotient

The goal of this note is to provide a general lower bound on the number of even values of the Fourier coefficients of an arbitrary eta-quotient [Formula: see text], over any arithmetic progression. Namely, if [Formula: see text] denotes the number of even coefficients of [Formula: see text] in degrees [Formula: see text] (mod [Formula: see text]) such that [Formula: see text], then we show that [Formula: see text] is unbounded for [Formula: see text] large. Note that our result is very close to the best bound currently known even in the special case of the partition function [Formula: see text] (namely, [Formula: see text], proven by Bellaïche and Nicolas in 2016). Our argument substantially relies upon, and generalizes, Serre’s classical theorem on the number of even values of [Formula: see text], combined with a recent modular-form result by Cotron et al. on the lacunarity modulo 2 of certain eta-quotients. Interestingly, even in the case of [Formula: see text] first shown by Serre, no elementary proof is known of this bound. At the end, we propose an elegant problem on quadratic representations, whose solution would finally yield a modular form-free proof of Serre’s theorem.
A Real-Time Fusion Framework for Long-term Visual Localization

Visual localization is a fundamental task that regresses the 6 Degree Of Freedom (6DoF) poses with image features in order to serve the high precision localization requests in many robotics applications. Degenerate conditions like motion blur, illumination changes and environment variations place great challenges in this task. Fusion with additional information, such as sequential information and Inertial Measurement Unit (IMU) inputs, would greatly assist such problems. In this paper, we present an efficient client-server visual localization architecture that fuses global and local pose estimations to realize promising precision and efficiency. We include additional geometry hints in mapping and global pose regressing modules to improve the measurement quality. A loosely coupled fusion policy is adopted to leverage the computation complexity and accuracy. We conduct the evaluations on two typical open-source benchmarks, 4Seasons and OpenLORIS. Quantitative results prove that our framework has competitive performance with respect to other state-of-the-art visual localization solutions.
Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition

Face recognition systems are widely deployed in safety-critical applications, including law enforcement, yet they exhibit bias across a range of socio-demographic dimensions, such as gender and race. Conventional wisdom dictates that model biases arise from biased training data. As a consequence, previous works on bias mitigation largely focused on pre-processing the training data, adding penalties to prevent bias from effecting the model during training, or post-processing predictions to debias them, yet these approaches have shown limited success on hard problems such as face recognition. In our work, we discover that biases are actually inherent to neural network architectures themselves. Following this reframing, we conduct the first neural architecture search for fairness, jointly with a search for hyperparameters. Our search outputs a suite of models which Pareto-dominate all other high-performance architectures and existing bias mitigation methods in terms of accuracy and fairness, often by large margins, on the two most widely used datasets for face identification, CelebA and VGGFace2. Furthermore, these models generalize to other datasets and sensitive attributes. We release our code, models and raw data files at https://github.com/dooleys/FR-NAS.
Concentration inequalities for Paley-Wiener spaces

This article considers the question of how much of the mass of an element in a Paley-Wiener space can be concentracted on a given set. We seek bounds in terms of relative densities of the given set. We extend a result of Donoho and Logan from 1992 in one dimension and consider similar results in higher dimensions.
Real-Time Driver Monitoring Systems through Modality and View Analysis

Driver distractions are known to be the dominant cause of road accidents. While monitoring systems can detect non-driving-related activities and facilitate reducing the risks, they must be accurate and efﬁcient to be appli-cable. Unfortunately, state-of-the-art methods prioritize accuracy while ignoring latency because they leverage cross-view and multimodal videos in which consecutive frames are highly similar. Thus, in this paper, we pursue time-effective detection models by neglecting the temporal relation between video frames and investigate the im-portance of each sensing modality in detecting drives’ activities. Experiments demonstrate that 1) our proposed algorithms are real-time and can achieve similar performances (97.5% AUC-PR) with signiﬁcantly reduced computation compared with video-based models; 2) the top view with the infrared channel is more informative than any other single modality. Furthermore, we enhance the DAD dataset by manually annotating its test set to enable multiclassiﬁcation. We also thoroughly analyze the inﬂu-ence of visual sensor types and their placements on the prediction of each class. The code and the new labels will be released.
An SPDE with Robin-type boundary for a system of elastically killed diffusions on the positive half-line

We consider a system of particles undergoing correlated diffusion with elastic boundary conditions on the half-line. By taking the large particle limit we establish existence and uniqueness for the limiting empirical measure valued process for the surviving particles. This process can be viewed as the weak form for an SPDE with a noisy Robin boundary condition satisfied by the particle density. We establish results on the $L^2$-regularity properties of this density process, showing that it is well behaved in the interior of the domain but may exhibit singularities on the boundary at a dense set of times. We also show existence of limit points for the empirical measure in the non-linear case where the particles have a measure dependent drift. We make connections for our linear problem to the corresponding absorbing and reflecting SPDEs, as the elastic parameter takes its extreme values.
Contextual bandits with concave rewards, and an application to fair ranking

We consider Contextual Bandits with Concave Rewards (CBCR), a multi-objective bandit problem where the desired trade-off between the rewards is defined by a known concave objective function, and the reward vector depends on an observed stochastic context. We present the first algorithm with provably vanishing regret for CBCR without restrictions on the policy space, whereas prior works were restricted to finite policy spaces or tabular representations. Our solution is based on a geometric interpretation of CBCR algorithms as optimization algorithms over the convex set of expected rewards spanned by all stochastic policies. Building on Frank-Wolfe analyses in constrained convex optimization, we derive a novel reduction from the CBCR regret to the regret of a scalar-reward bandit problem. We illustrate how to apply the reduction off-the-shelf to obtain algorithms for CBCR with both linear and general reward functions, in the case of non-combinatorial actions. Motivated by fairness in recommendation, we describe a special case of CBCR with rankings and fairness-aware objectives, leading to the first algorithm with regret guarantees for contextual combinatorial bandits with fairness of exposure.
Morig: Motion-Aware Rigging of Character Meshes from Point Clouds

We present MoRig, a method that automatically rigs character meshes driven by single-view point cloud streams capturing the motion of performing characters. Our method is also able to animate the 3D meshes according to the captured point cloud motion. MoRig’s neural network encodes motion cues from the point clouds into features that are informative about the articulated parts of the performing character. These motion-aware features guide the inference of an appropriate skeletal rig for the input mesh, which is then animated based on the point cloud motion. Our method can rig and animate diverse characters, including humanoids, quadrupeds, and toys with varying articulation. It accounts for occluded regions in the point clouds and mismatches in the part proportions between the input mesh and captured character. Compared to other rigging approaches that ignore motion cues, MoRig produces more accurate rigs, well-suited for re-targeting motion from captured characters.
Differentially Private Diffusion Models

While modern machine learning models rely on increasingly large training datasets, data is often limited in privacy-sensitive domains. Generative models trained with differential privacy (DP) on sensitive data can sidestep this challenge, providing access to synthetic data instead. We build on the recent success of diffusion models (DMs) and introduce Differentially Private Diffusion Models (DPDMs), which enforce privacy using differentially private stochastic gradient descent (DP-SGD). We investigate the DM parameterization and the sampling algorithm, which turn out to be crucial ingredients in DPDMs, and propose noise multiplicity, a powerful modification of DP-SGD tailored to the training of DMs. We validate our novel DPDMs on image generation benchmarks and achieve state-of-the-art performance in all experiments. Moreover, on standard benchmarks, classifiers trained on DPDM-generated synthetic data perform on par with task-specific DP-SGD-trained classifiers, which has not been demonstrated before for DP generative models. Project page and code: https://nv-tlabs.github. io/DPDM.
Probability Weighted Clustered Coefficients Regression Models in Complex Survey Sampling

Regression analysis is commonly conducted in survey sampling. However, existing methods fail when the relationships vary across different areas or domains. In this paper, we propose a unified framework to study the group-wise covariate effect under complex survey sampling based on pairwise penalties, and the associated objective function is solved by the alternating direction method of multipliers. Theoretical properties of the proposed method are investigated under some generality conditions. Numerical experiments demonstrate the superiority of the proposed method in terms of identifying groups and estimation efficiency for both linear regression models and logistic regression models.
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks

How reliably can we trust the scores obtained from social bias benchmarks as faithful indicators of problematic social biases in a given model? In this work, we study this question by contrasting social biases with non-social biases that stem from choices made during dataset construction (which might not even be discernible to the human eye). To do so, we empirically simulate various alternative constructions for a given benchmark based on seemingly innocuous modifications (such as paraphrasing or random-sampling) that maintain the essence of their social bias. On two well-known social bias benchmarks (Winogender and BiasNLI), we observe that these shallow modifications have a surprising effect on the resulting degree of bias across various models and consequently the relative ordering of these models when ranked by measured bias. We hope these troubling observations motivate more robust measures of social biases.
Modelling Large Dimensional Datasets with Markov Switching Factor Models

We study a novel large dimensional approximate factor model with regime changes in the loadings driven by a latent first order Markov process. By exploiting the equivalent linear representation of the model, we first recover the latent factors by means of Principal Component Analysis. We then cast the model in state-space form, and we estimate loadings and transition probabilities through an EM algorithm based on a modified version of the Baum-Lindgren-Hamilton-Kim filter and smoother that makes use of the factors previously estimated. Our approach is appealing as it provides closed form expressions for all estimators. More importantly, it does not require knowledge of the true number of factors. We derive the theoretical properties of the proposed estimation procedure, and we show their good finite sample performance through a comprehensive set of Monte Carlo experiments. The empirical usefulness of our approach is illustrated through an application to a large portfolio of stocks.
Deep Bidirectional Language-Knowledge Graph Pretraining

Pretraining a language model (LM) on text has been shown to help various downstream NLP tasks. Recent works show that a knowledge graph (KG) can complement text data, offering structured background knowledge that provides a useful scaffold for reasoning. However, these works are not pretrained to learn a deep fusion of the two modalities at scale, limiting the potential to acquire fully joint representations of text and KG. Here we propose DRAGON (Deep Bidirectional Language-Knowledge Graph Pretraining), a self-supervised approach to pretraining a deeply joint language-knowledge foundation model from text and KG at scale. Specifically, our model takes pairs of text segments and relevant KG subgraphs as input and bidirectionally fuses information from both modalities. We pretrain this model by unifying two self-supervised reasoning tasks, masked language modeling and KG link prediction. DRAGON outperforms existing LM and LM+KG models on diverse downstream tasks including question answering across general and biomedical domains, with +5% absolute gain on average. In particular, DRAGON achieves notable performance on complex reasoning about language and knowledge (+10% on questions involving long contexts or multi-step reasoning) and low-resource QA (+8% on OBQA and RiddleSense), and new state-of-the-art results on various BioNLP tasks. Our code and trained models are available at https://github.com/michiyasunaga/dragon.