Miracle-dz/UniFAR-FTUs huggingface dataset:
Exploring Adversarial Learning for Deep Semi-Supervised Facial Action Unit Recognition

Current works formulate facial action unit (AU) recognition as a supervised learning problem, requiring fully AU-labeled facial images during training. It is challenging if not impossible to provide AU annotations for large numbers of facial images. Fortunately, AUs appear on all facial images, whether manually labeled or not, satisfy the underlying anatomic mechanisms and human behavioral habits. In this paper, we propose a deep semi-supervised framework for facial action unit recognition from partially AU-labeled facial images. Specifically, the proposed deep semi-supervised AU recognition approach consists of a deep recognition network and a discriminator D. The deep recognition network R learns facial representations from large-scale facial images and AU classifiers from limited ground truth AU labels. The discriminator D is introduced to enforce statistical similarity between the AU distribution inherent in ground truth AU labels and the distribution of the predicted AU labels from labeled and unlabeled facial images. The deep recognition network aims to minimize recognition loss from the labeled facial images, to faithfully represent inherent AU distribution for both labeled and unlabeled facial images, and to confuse the discriminator. During training, the deep recognition network R and the discriminator D are optimized alternately. Thus, the inherent AU distributions caused by underlying anatomic mechanisms are leveraged to construct better feature representations and AU classifiers from partially AU-labeled data during training. Experiments on two benchmark databases demonstrate that the proposed approach successfully captures AU distributions through adversarial learning and outperforms state-of-the-art AU recognition work.
You Can't Always Get What You Want: Towards User-Controlled Privacy on Android

Mobile applications (hereafter, apps) collect a plethora of information regarding the user behavior and his device through third-party analytics libraries. However, the collection and usage of such data raised several privacy concerns, mainly because the end-user - i.e., the actual owner of the data - is out of the loop in this collection process. Also, the existing privacy-enhanced solutions that emerged in the last years follow an ”all or nothing” approach, leaving the user the sole option to accept or completely deny access to privacy-related data. This work has the two-fold objective of assessing the privacy impact of mobile analytics libraries and proposing a data anonymization methodology that offers a trade-off between the utility and privacy of the collected data and enables complete control over the sharing process. To achieve that, we present an empirical privacy assessment on the analytics libraries used in the 4500 most-used Android apps of the Google Play Store in late 2020. Then, we propose an empowered anonymization methodology, based on MobHide (Caputo et al., 2020), that gives the end-user complete control over the collection and anonymization process. Finally, we empirically demonstrate the applicability and effectiveness of our solution thanks to HideDroid, a fully-fledged anonymization app for the Android ecosystem.
ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure

Curvature in form of the Hessian or its generalized Gauss-Newton (GGN) approximation is valuable for algorithms that rely on a local model for the loss to train, compress, or explain deep networks. Existing methods based on implicit multiplication via automatic differentiation or Kronecker-factored block diagonal approximations do not consider noise in the mini-batch. We present ViViT, a curvature model that leverages the GGN's low-rank structure without further approximations. It allows for efficient computation of eigenvalues, eigenvectors, as well as per-sample first- and second-order directional derivatives. The representation is computed in parallel with gradients in one backward pass and offers a fine-grained cost-accuracy trade-off, which allows it to scale. We demonstrate this by conducting performance benchmarks and substantiate ViViT's usefulness by studying the impact of noise on the GGN's structural properties during neural network training.
Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey

This is a tutorial and survey paper for nonlinear dimensionality and feature extraction methods which are based on the Laplacian of graph of data. We ﬁrst introduce adjacency matrix, deﬁnition of Laplacian matrix, and the interpretation of Laplacian. Then, we cover the cuts of graph and spectral clustering which applies clustering in a subspace of data. Different optimization variants of Laplacian eigenmap and its out-of-sample extension are explained. Thereafter, we introduce the locality preserving projection and its kernel variant as linear special cases of Laplacian eigenmap. Versions of graph embedding are then explained which are generalized versions of Laplacian eigenmap and locality preserving projection. Finally, diffusion map is introduced which is a method based on Laplacian of data and random walks on the data graph.
Visible-light Phase Curves from the Second Year of the TESS Primary Mission

We carried out a systematic study of full-orbit phase curves for known transiting systems in the northern ecliptic sky that were observed during Year 2 of the TESS primary mission. We applied the same methodology for target selection, data processing, and light-curve fitting as we did in our Year 1 study. Out of the 15 transiting systems selected for analysis, seven—HAT-P-7, KELT-1, KELT-9, KELT-16, KELT-20, Kepler-13A, and WASP-12—show statistically significant secondary eclipses and day–night atmospheric brightness modulations. Small eastward dayside hot-spot offsets were measured for KELT-9b and WASP-12b. KELT-1, Kepler-13A, and WASP-12 show additional phase-curve variability attributed to the tidal distortion of the host star; the amplitudes of these signals are consistent with theoretical predictions. We combined occultation measurements from TESS and Spitzer to compute dayside brightness temperatures, TESS-band geometric albedos, Bond albedos, and phase integrals for several systems. The new albedo values solidify the previously reported trend between dayside temperature and geometric albedo for planets with 1500 K < T day < 3000 K. For Kepler-13Ab, we carried out an atmospheric retrieval of the full secondary eclipse spectrum, which revealed a noninverted temperature–pressure profile, significant H2O and K absorption in the near-infrared, evidence for strong optical atmospheric opacity due to sodium, and a confirmation of the high geometric albedo inferred from our simpler analysis. We explore the implications of the phase integrals (ratios of Bond to geometric albedos) for understanding exoplanet clouds. We also report updated transit ephemerides for all of the systems studied in this work.
A Database for Research on Detection and Enhancement of Speech Transmitted over HF links

In this paper we present an open database for the development of detection and enhancement algorithms of speech transmitted over HF radio channels. It consists of audio samples recorded by various receivers at different locations across Europe, all monitoring the same single-sideband modulated transmission from a base station in Paderborn, Germany. Transmitted and received speech signals are precisely time aligned to offer parallel data for supervised training of deep learning based detection and enhancement algorithms. For the task of speech activity detection two exemplary baseline systems are presented, one based on statistical methods employing a multi-stage Wiener filter with minimum statistics noise floor estimation, and the other relying on a deep learning approach.
Gravitational collapse from cold uniform asymmetric initial conditions

Using controlled numerical N-body experiments, we show how, in the collapse dynamics of an initially cold and uniform distribution of particles with a generic asymmetric shape, finite N fluctuations and perturbations induced by the anisotropic gravitational field compete to determine the physical properties of the asymptotic quasi-stationary state. When finite N fluctuations dominate the dynamics, the particle energy distribution changes greatly and the final density profile decays outside its core as r−4 with an N-dependent amplitude. On the other hand, in the limit where the anisotropic perturbations dominate, the collapse is softer and the density profile shows a decay as r−3, as is typical of halos in cosmological simulations. However, even in this limit, convergence with N of the macroscopic properties of the virialized system, such as the particle energy distributions, the bound mass, and the density profile, is very slow and not clearly established, including for our largest simulations (with N ∼ 106). Our results illustrate the challenges of accurately simulating the first collapsing structures in standard-type cosmological models.
COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion

Despite recent successes of large pre-trained language models in solving reasoning tasks, their inference capabilities remain opaque. We posit that such models can be made more interpretable by explicitly generating interim inference rules, and using them to guide the generation of task-specific textual outputs. In this paper we present Coins, a recursive inference framework that i) iteratively reads context sentences, ii) dynamically generates contextualized inference rules, encodes them, and iii) uses them to guide task-specific output generation. We apply to a Narrative Story Completion task that asks a model to complete a story with missing sentences, to produce a coherent story with plausible logical connections, causal relationships, and temporal dependencies. By modularizing inference and sentence generation steps in a recurrent model, we aim to make reasoning steps and their effects on next sentence generation transparent. Our automatic and manual evaluations show that the model generates better story sentences than SOTA baselines, especially in terms of coherence. We further demonstrate improved performance over strong pre-trained LMs in generating commonsense inference rules. The recursive nature of holds the potential for controlled generation of longer sequences.
Semi-Empirical Objective Functions for MCMC Proposal Optimization

Current objective functions used for training neural MCMC proposal distributions implicitly rely on architectural restrictions to yield sensible optimization results, which hampers the development of highly expressive neural MCMC proposal architectures. In this work, we introduce and demonstrate a semi-empirical procedure for determining approximate objective functions suitable for optimizing arbitrarily parameterized proposal distributions in MCMC methods. Our proposed Ab Initio objective functions consist of the weighted combination of functions following constraints on their global optima and transformation invariances that we argue should be upheld by general measures of MCMC efficiency for use in proposal optimization. Our experimental results demonstrate that Ab Initio objective functions maintain favorable performance and preferable optimization behavior compared to existing objective functions for neural MCMC optimization. We find that Ab Initio objective functions are sufficiently robust to enable the confident optimization of neural proposal distributions parameterized by deep generative networks extending beyond the regimes of traditional MCMC schemes.
A Survey on Deep Domain Adaptation for LiDAR Perception

Scalable systems for automated driving have to reliably cope with an open-world setting. This means, the perception systems are exposed to drastic domain shifts, like changes in weather conditions, time-dependent aspects, or geographic regions. Covering all domains with annotated data is impossible because of the endless variations of domains and the time-consuming and expensive annotation process. Furthermore, fast development cycles of the system additionally introduce hardware changes, such as sensor types and vehicle setups, and the required knowledge transfer from simulation.To enable scalable automated driving, it is therefore crucial to address these domain shifts in a robust and efficient manner. Over the last years, a vast amount of different domain adaptation techniques evolved. There already exists a number of survey papers for domain adaptation on camera images, however, a survey for LiDAR perception is absent. Nevertheless, LiDAR is a vital sensor for automated driving that provides detailed 3D scans of the vehicle’s surroundings. To stimulate future research, this paper presents a comprehensive review of recent progress in domain adaptation methods and formulates interesting research questions specifically targeted towards LiDAR perception.
NMS-Loss: Learning with Non-Maximum Suppression for Crowded Pedestrian Detection

Non-Maximum Suppression (NMS) is essential for object detection and affects the evaluation results by incorporating False Positives (FP) and False Negatives (FN), especially in crowd occlusion scenes. In this paper, we raise the problem of weak connection between the training targets and the evaluation metrics caused by NMS and propose a novel NMS-Loss making the NMS procedure can be trained end-to-end without any additional network parameters. Our NMS-Loss punishes two cases when FP is not suppressed and FN is wrongly eliminated by NMS. Specifically, we propose a pull loss to pull predictions with the same target close to each other, and a push loss to push predictions with different targets away from each other. Experimental results show that with the help of NMS-Loss, our detector, namely NMS-Ped, achieves impressive results with Miss Rate of 5.92% on Caltech dataset and 10.08%on CityPersons dataset, which are both better than state-of-the-art competitors.
Multitask Online Mirror Descent

We introduce and analyze MT-OMD, a multitask generalization of Online Mirror Descent (OMD) which operates by sharing updates between tasks. We prove that the regret of MT-OMD is of order $\sqrt{1 + \sigma^2(N-1)}\sqrt{T}$, where $\sigma^2$ is the task variance according to the geometry induced by the regularizer, $N$ is the number of tasks, and $T$ is the time horizon. Whenever tasks are similar, that is $\sigma^2 \le 1$, our method improves upon the $\sqrt{NT}$ bound obtained by running independent OMDs on each task. We further provide a matching lower bound, and show that our multitask extensions of Online Gradient Descent and Exponentiated Gradient, two major instances of OMD, enjoy closed-form updates, making them easy to use in practice. Finally, we present experiments which support our theoretical findings.
Matrix Completion With Data-Dependent Missingness Probabilities

The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number <inline-formula> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> such that each entry of the matrix is available independently with probability <inline-formula> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function <inline-formula> <tex-math notation="LaTeX">$f$ </tex-math></inline-formula> of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function <inline-formula> <tex-math notation="LaTeX">$f$ </tex-math></inline-formula>.
The Willmore flow with prescribed isoperimetric ratio

We introduce a non-local $L^2$-gradient flow for the Willmore energy of immersed surfaces which preserves the isoperimetric ratio. For spherical initial data with energy below an explicit threshold, we show long-time existence and convergence to a Helfrich immersion. This is in sharp contrast to the locally constrained flow, where finite time singularities occur.
When does the Lanczos algorithm compute exactly?

In theory, the Lanczos algorithm generates an orthogonal basis of the corresponding Krylov subspace. However, in finite precision arithmetic, the orthogonality and linear independence of the computed Lanczos vectors is usually lost quickly. In this paper we study a class of matrices and starting vectors having a special nonzero structure that guarantees exact computations of the Lanczos algorithm whenever floating point arithmetic satisfying the IEEE 754 standard is used. Analogous results are formulated also for a variant of the conjugate gradient method that produces then almost exact results. The results are extended to the Arnoldi algorithm, the nonsymmetric Lanczos algorithm, the Golub-Kahan bidiagonalization, the block-Lanczos algorithm and their counterparts for solving linear systems.
MERLOT: Multimodal Neural Script Knowledge Models

As humans, we understand events in the visual world contextually, performing multimodal reasoning across time to make inferences about the past, present, and future. We introduce MERLOT, a model that learns multimodal script knowledge by watching millions of YouTube videos with transcribed speech -- in an entirely label-free, self-supervised manner. By pretraining with a mix of both frame-level (spatial) and video-level (temporal) objectives, our model not only learns to match images to temporally corresponding words, but also to contextualize what is happening globally over time. As a result, MERLOT exhibits strong out-of-the-box representations of temporal commonsense, and achieves state-of-the-art performance on 12 different video QA datasets when finetuned. It also transfers well to the world of static images, allowing models to reason about the dynamic context behind visual scenes. On Visual Commonsense Reasoning, MERLOT answers questions correctly with 80.6% accuracy, outperforming state-of-the-art models of similar size by over 3%, even those that make heavy use of auxiliary supervised data (like object bounding boxes). Ablation analyses demonstrate the complementary importance of: 1) training on videos versus static images; 2) scaling the magnitude and diversity of the pretraining video corpus; and 3) using diverse objectives that encourage full-stack multimodal reasoning, from the recognition to cognition level.
Parallel and External-Memory Construction of Minimal Perfect Hash Functions With PTHash

A function <inline-formula><tex-math notation="LaTeX">$f : U \to \lbrace 0,\ldots,n-1\rbrace$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>f</mml:mi><mml:mo>:</mml:mo><mml:mi>U</mml:mi><mml:mo>→</mml:mo><mml:mo>{</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mo>. </mml:mo><mml:mo>,</mml:mo><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn><mml:mo>}</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="pibiri-ieq1-3303341.gif"/></alternatives></inline-formula> is a <italic>minimal perfect hash function</italic> for a set <inline-formula><tex-math notation="LaTeX">$S \subseteq U$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>S</mml:mi><mml:mo>⊆</mml:mo><mml:mi>U</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="pibiri-ieq2-3303341.gif"/></alternatives></inline-formula> of size <inline-formula><tex-math notation="LaTeX">$n$</tex-math><alternatives><mml:math><mml:mi>n</mml:mi></mml:math><inline-graphic xlink:href="pibiri-ieq3-3303341.gif"/></alternatives></inline-formula>, if <inline-formula><tex-math notation="LaTeX">$f$</tex-math><alternatives><mml:math><mml:mi>f</mml:mi></mml:math><inline-graphic xlink:href="pibiri-ieq4-3303341.gif"/></alternatives></inline-formula> bijectively maps <inline-formula><tex-math notation="LaTeX">$S$</tex-math><alternatives><mml:math><mml:mi>S</mml:mi></mml:math><inline-graphic xlink:href="pibiri-ieq5-3303341.gif"/></alternatives></inline-formula> into the first <inline-formula><tex-math notation="LaTeX">$n$</tex-math><alternatives><mml:math><mml:mi>n</mml:mi></mml:math><inline-graphic xlink:href="pibiri-ieq6-3303341.gif"/></alternatives></inline-formula> natural numbers. These functions are important for many practical applications in computing, such as search engines, computer networks, and databases. Several algorithms have been proposed to build minimal perfect hash functions that: scale well to large sets, retain fast evaluation time, and take very little space, e.g., 2 – 3 bits/key. PTHash is one such algorithm, achieving very fast evaluation in compressed space, typically many times faster than other techniques. In this work, we propose a new construction algorithm for PTHash enabling: (1) <italic>multi-threading</italic>, to either build functions more quickly or more space-efficiently, and (2) <italic>external-memory processing</italic>, to scale to inputs much larger than the available internal memory. Only few other algorithms in the literature share these features, despite of their practical impact. We conduct an extensive experimental assessment on large real-world string collections and show that, with respect to other techniques, PTHash is competitive in construction time and space consumption, but retains 2 – 6× better lookup time.
Great Service! Fine-grained Parsing of Implicit Arguments

Broad-coverage meaning representations in NLP mostly focus on explicitly expressed content. More importantly, the scarcity of datasets annotating diverse implicit roles limits empirical studies into their linguistic nuances. For example, in the web review “Great service! ”, the provider and consumer are implicit arguments of different types. We examine an annotated corpus of fine-grained implicit arguments (Cui and Hershcovich, 2020) by carefully re-annotating it, resolving several inconsistencies. Subsequently, we present the first transition-based neural parser that can handle implicit arguments dynamically, and experiment with two different transition systems on the improved dataset. We find that certain types of implicit arguments are more difficult to parse than others and that the simpler system is more accurate in recovering implicit arguments, despite having a lower overall parsing score, attesting current reasoning limitations of NLP models. This work will facilitate a better understanding of implicit and underspecified language, by incorporating it holistically into meaning representations.