Miracle-dz/UniFAR-FTUs huggingface dataset:
LFPS-Net: a lightweight fast pulse simulation network for BVP estimation

—Heart rate estimation based on remote photoplethys-mography plays an important role in several speciﬁc scenarios, such as health monitoring and fatigue detection. Existing well-established methods are committed to taking the average of the predicted HRs of multiple overlapping video clips as the ﬁnal re-sults for the 30-second facial video. Although these methods with hundreds of layers and thousands of channels are highly accurate and robust, they require enormous computational budget and a 30-second wait time, which greatly limits the application of the algorithms to scale. Under these cicumstacnces, We propose a lightweight fast pulse simulation network (LFPS-Net), pursuing the best accuracy within a very limited computational and time budget, focusing on common mobile platforms, such as smart phones. In order to suppress the noise component and get stable pulse in a short time, we design a multi-frequency modal signal fusion mechanism, which exploits the theory of time-frequency domain analysis to separate multi-modal information from complex signals. It helps proceeding network learn the effective fetures more easily without adding any parameter. In addition, we design a oversampling training strategy to solve the problem caused by the unbalanced distribution of dataset. For the 30-second facial videos, our proposed method achieves the best results on most of the evaluation metrics for estimating heart rate or heart rate variability compared to the best available papers. The proposed method can still obtain very competitive results by using a short-time ( 15-second) facail video.
Regularized modified log-Sobolev inequalities, and comparison of Markov chains

In this work, we develop a comparison procedure for the Modiﬁed log-Sobolev Inequality (MLSI) constants of two reversible Markov chains on a ﬁnite state space. Eﬃcient comparison of the MLSI Dirichlet forms is a well known obstacle in the theory of Markov chains. We approach this problem by introducing a regularized MLSI constant which, under some assumptions, has the same order of magnitude as the usual MLSI constant yet is amenable for comparison and thus considerably simpler to estimate in certain cases. As an application of this general comparison procedure, we provide a sharp estimate of the MLSI constant of the switch chain on the the set of simple bipartite regular graphs of size n with a ﬁxed degree d . Our estimate implies that the total variation mixing time of the switch chain is of order O d ( n log n ). The result is optimal up to a multiple depending on d and resolves a long-standing open problem. We expect that the MLSI comparison technique implemented in this paper will ﬁnd further applications.
Ev-NeRF: Event Based Neural Radiance Field

We present Ev-NeRF, a Neural Radiance Field derived from event data. While event cameras can measure subtle brightness changes in high frame rates, the measurements in low lighting or extreme motion suffer from significant domain discrepancy with complex noise. As a result, the performance of event-based vision tasks does not transfer to challenging environments, where the event cameras are expected to thrive over normal cameras. We find that the multi-view consistency of NeRF provides a powerful self-supervision signal for eliminating spurious measurements and extracting the consistent underlying structure despite highly noisy input. Instead of posed images of the original NeRF, the input to Ev-NeRF is the event measurements accompanied by the movements of the sensors. Using the loss function that reflects the measurement model of the sensor, Ev-NeRF creates an integrated neural volume that summarizes the unstructured and sparse data points captured for about 2-4 seconds. The generated neural volume can also produce intensity images from novel views with reasonable depth estimates, which can serve as a high-quality input to various vision-based tasks. Our results show that Ev-NeRF achieves competitive performance for intensity image reconstruction under extreme noise and high-dynamic-range imaging.
Learning to Infer 3D Shape Programs with Differentiable Renderer

Given everyday artifacts, such as tables and chairs, humans recognize high-level regularities within them, such as the symmetries of a table, the repetition of its legs, while possessing low-level priors of their geometries, e.g., surfaces are smooth and edges are sharp. This kind of knowledge constitutes an important part of human perceptual understanding and reasoning. Representations of and how to reason in such knowledge, and the acquisition thereof, are still open questions in artiﬁcial intelligence (AI) and cognitive science. Building on the previous proposal of the 3D shape programs representation alone with the accompanying neural generator and executor from Tian et al. [12], we propose an analytical yet differentiable executor that is more faithful and controllable in interpreting shape programs (particularly in extrapolation) and more sample efﬁcient (requires no training). These facilitate the generator’s learning when ground truth programs are not available, and should be especially useful when new shape-program components are enrolled either by human designers or—in the context of library learning— algorithms themselves. Preliminary experiments on using it for adaptation illustrate the aforesaid advantages of the proposed module, encouraging similar methods being explored in building machines that learn to reason with the kind of knowledge described above, and even learn this knowledge itself.
A Topic-Attentive Transformer-based Model For Multimodal Depression Detection

Depression is one of the most common mental disorders, which imposes heavy negative impacts on one's daily life. Diagnosing depression based on the interview is usually in the form of questions and answers. In this process, the audio signals and their text transcripts of a subject are correlated to depression cues and easily recorded. Therefore, it is feasible to build an Automatic Depression Detection (ADD) model based on the data of these modalities in practice. However, there are two major challenges that should be addressed for constructing an effective ADD model. The first challenge is the organization of the textual and audio data, which can be of various contents and lengths for different subjects. The second challenge is the lack of training samples due to the privacy concern. Targeting to these two challenges, we propose the TOpic ATtentive transformer-based ADD model, abbreviated as TOAT. To address the first challenge, in the TOAT model, topic is taken as the basic unit of the textual and audio data according to the question-answer form in a typical interviewing process. Based on that, a topic attention module is designed to learn the importance of of each topic, which helps the model better retrieve the depressed samples. To solve the issue of data scarcity, we introduce large pre-trained models, and the fine-tuning strategy is adopted based on the small-scale ADD training data. We also design a two-branch architecture with a late-fusion strategy for building the TOAT model, in which the textual and audio data are encoded independently. We evaluate our model on the multimodal DAIC-WOZ dataset specifically designed for the ADD task. Experimental results show the superiority of our method. More importantly, the ablation studies demonstrate the effectiveness of the key elements in the TOAT model.
Video2StyleGAN: Encoding Video in Latent Space for Manipulation

Many recent works have been proposed for face image editing by leveraging the latent space of pretrained GANs. However, few attempts have been made to directly apply them to videos, because 1) they do not guarantee temporal consistency, 2) their application is limited by their processing speed on videos, and 3) they cannot accurately encode details of face motion and expression. To this end, we propose a novel network to encode face videos into the latent space of StyleGAN for semantic face video manipulation. Based on the vision transformer, our network reuses the high-resolution portion of the latent vector to enforce temporal consistency. To capture subtle face motions and expressions, we design novel losses that involve sparse facial landmarks and dense 3D face mesh. We have thoroughly evaluated our approach and successfully demonstrated its application to various face video manipulations. Particularly, we propose a novel network for pose/expression control in a 3D coordinate system. Both qualitative and quantitative results have shown that our approach can significantly outperform existing single image methods, while achieving real-time (66 fps) speed.
Insights Into Deep Non-Linear Filters for Improved Multi-Channel Speech Enhancement

The key advantage of using multiple microphones for speech enhancement is that spatial filtering can be used to complement the tempo-spectral processing. In a traditional setting, linear spatial filtering (beamforming) and single-channel post-filtering are commonly performed separately. In contrast, there is a trend towards employing (DNNs) to learn a joint spatial and tempo-spectral non-linear filter, which means that the restriction of a linear processing model and that of a separate processing of spatial and tempo-spectral information can potentially be overcome. However, the internal mechanisms that lead to good performance of such data-driven filters for multi-channel speech enhancement are not well understood. Therefore, in this work, we analyse the properties of a non-linear spatial filter realized by a DNN as well as its interdependency with temporal and spectral processing by carefully controlling the information sources (spatial, spectral, and temporal) available to the network. We confirm the superiority of a non-linear spatial processing model, which outperforms an oracle linear spatial filter in a challenging speaker extraction scenario for a low number of microphones by 0.24 POLQA score. Our analyses reveal that in particular spectral information should be processed jointly with spatial information as this increases the spatial selectivity of the filter. Our systematic evaluation then leads to a simple network architecture, that outperforms state-of-the-art network architectures on a speaker extraction task by 0.22 POLQA score and by 0.32 POLQA score on the CHiME3 data.
Mobility State Detection of Cellular-Connected UAVs Based on Handover Count Statistics

Estimating the speed of aerial user equipment (UE) is critically important to provide reliable mobility management for cellular-connected unmanned aerial vehicles (UAVs) since this can enhance the quality of service. The 3GPP standard uses the number of handovers made by a UE during a predefined time period to estimate the speed and the mobility state efficiently. In this article, we introduce an approximation to the probability mass function of handover count (HOC) as a function of a cellular-connected UAV's height and velocity, HOC measurement time window, and different ground base station (GBS) densities. Afterward, we derive the Cramer-Rao lower bound (CRLB) for the speed estimate of a UAV and also provide a simple biased estimator for its speed based on the GBS density and HOC measurement period. We show that for a low handover parameter, the biased estimator turns into a minimum variance unbiased estimator (MVUE). Using this estimator, we study the problem of detecting the mobility state of a UAV as low, medium, or high mobility as per the 3GPP specifications. Using our proposed MVUE, we also characterize the accuracy improvement in speed estimation and mobility state detection as the GBS density and the HOC measurement window increase.
Maximum independent set (stable set) problem: A mathematical programming model with valid inequalities and computational testing

This paper deals with the maximum independent set (M.I.S.) problem, also known as the stable set problem. The basic mathematical programming model that captures this problem is an Integer Program (I.P.) with zero-one variables and only the edge inequalities . We present an enhanced model by adding a polynomial number of linear constraints, known as valid inequalities ; this new model is still polynomial in the number of vertices in the graph. We carried out computational testing of the Linear Relaxation of the new Integer Program. We tested about 7000 instances of randomly generated (and connected) graphs with up to 64 vertices. In each of these instances, the Linear Relaxation returned an optimal solution with (i) every variable having an integer value, and (ii) the optimal solution value of the Linear Relaxation was the same as that of the original ( basic ) Integer Program. For certain instances, a binary search on the objective function value is a useful tool which yields a (weakly) polynomial algorithm.
Field change for the Cassels-Tate pairing and applications to class groups

A BSTRACT . In [19], we deﬁned a category SMod F of ﬁnite Galois modules decorated with local conditions for each global ﬁeld F . In this paper, given an extension K/F of global ﬁelds, we deﬁne a restriction of scalars functor from SMod K to SMod F and show that it behaves well with respect to the Cassels–Tate pairing. We apply this work to study the class groups of global ﬁelds in the context of the Cohen–Lenstra heuristics.
Simplifying Semantic Annotations of SMCalFlow

SMCalFlow (Semantic Machines et al., 2020) is a large corpus of semantically detailed annotations of task-oriented natural dialogues. The annotations use a dataflow approach, in which the annotations are programs which represent user requests. Despite the availability, size and richness of this annotated corpus, it has seen only very limited use in dialogue systems research work, at least in part due to the difficulty in understanding and using the annotations. To address these difficulties, this paper suggests a simplification of the SMCalFlow annotations, as well as releases code needed to inspect the execution of the annotated dataflow programs, which should allow researchers of dialogue systems an easy entry point to experiment with various dataflow based implementations and annotations.
TEVR: Improving Speech Recognition by Token Entropy Variance Reduction

This paper presents TEVR, a speech recognition model designed to minimize the variation in token entropy w.r. t. to the language model. This takes advantage of the fact that if the language model will reliably and accurately predict a token anyway, then the acoustic model doesn't need to be accurate in recognizing it. We train German ASR models with 900 million parameters and show that on CommonVoice German, TEVR scores a very competitive 3.64% word error rate, which outperforms the best reported results by a relative 16.89% reduction in word error rate. We hope that releasing our fully trained speech recognition pipeline to the community will lead to privacy-preserving offline virtual assistants in the future.
The Bandwagon Effect: Not Just Another Bias

Optimizing recommender systems based on user interaction data is mainly seen as a problem of dealing with selection bias, where most existing work assumes that interactions from different users are independent. However, it has been shown that in reality user feedback is often influenced by earlier interactions of other users, e.g. via average ratings, number of views or sales per item, etc. This phenomenon is known as the bandwagon effect. In contrast with previous literature, we argue that the bandwagon effect should not be seen as a problem of statistical bias. In fact, we prove that this effect leaves both individual interactions and their sample mean unbiased. Nevertheless, we show that it can make estimators inconsistent, introducing a distinct set of problems for convergence in relevance estimation. Our theoretical analysis investigates the conditions under which the bandwagon effect poses a consistency problem and explores several approaches for mitigating these issues. This work aims to show that the bandwagon effect poses an underinvestigated open problem that is fundamentally distinct from the well-studied selection bias in recommendation.
Locked and unlocked smooth embeddings of surfaces

We study the continuous motion of smooth isometric embeddings of a planar surface in three-dimensional Euclidean space, and two related discrete analogues of these embeddings, polygonal embeddings and flat foldings without interior vertices, under continuous changes of the embedding or folding. We show that every star-shaped or spiral-shaped domain is unlocked: a continuous motion unfolds it to a flat embedding. However, disks with two holes can have locked embeddings that are topologically equivalent to a flat embedding but cannot reach a flat embedding by continuous motion.
Implementation of SquashFS Support in U-Boot

U-Boot is a notorious bootloader and Open Source project. This work had as objective adding support for the SquashFS filesystem to U-Boot and the support developed was submitted as a contribution to the project. The bootloader is responsible, in this context, for loading the kernel and the device tree blob into RAM. It needs to be capable of reading a storage device's partition formatted with a specific filesystem type. Adding this support allows U-Boot to read from SquashFS partitions. The source code was submitted to U-Boot's mailing list through a series of patches to be reviewed by one of the project's maintainer. Once it gets merged, the support will be used and modified by U-Boot's international community.
MMSE Symbol Level Precoding Under a Per Antenna Power Constraint for Multiuser MIMO Systems With PSK Modulation

This letter proposes two MMSE symbol-level precoding approaches considering a strict per antenna power constraint and PSK modulation for perfect and imperfect channel state information scenarios. The proposed designs are formulated as second-order cone programs, allowing for an optimal solution via the interior point method. Numerical results confirm that, for the scenario with perfect channel state information, the proposed designs outperform the existing techniques in terms of bit-error-rate for low and intermediate signal-to-noise-ratio. Numerical evaluations also confirm the superiority of the proposed robust MMSE design in the presence of channel state information imperfection.
Multilevel Bidirectional Cache Filter

Modern caches are often required to handle a massive amount of data, which exceeds the amount of available memory; thus, hybrid caches, specifically DRAM/SSD combination, become more and more prevalent. In such environments, in addition to the classical hit-ratio target, saving writes to the second-level cache is a dominant factor to avoid write amplification and wear out, two notorious phenomena of SSD. This paper presents BiDiFilter, a novel multilevel caching scheme that controls demotions and promotions between cache levels using a frequency sketch filter. Further, it splits the higher cache level into two areas to keep the most recent and the most frequent items close to the user. We conduct an extensive evaluation over real-world traces, comparing to previous multilevel policies. We show that using our mechanism yields an x10 saving of writes in almost all cases and often improving latencies by up to 20%.
Sum-of-Squares Relaxations for Information Theory and Variational Inference

We consider extensions of the Shannon relative entropy, referred to as $f$-divergences.Three classical related computational problems are typically associated with these divergences: (a) estimation from moments, (b) computing normalizing integrals, and (c) variational inference in probabilistic models. These problems are related to one another through convex duality, and for all them, there are many applications throughout data science, and we aim for computationally tractable approximation algorithms that preserve properties of the original problem such as potential convexity or monotonicity. In order to achieve this, we derive a sequence of convex relaxations for computing these divergences from non-centered covariance matrices associated with a given feature vector: starting from the typically non-tractable optimal lower-bound, we consider an additional relaxation based on ``sums-of-squares'', which is is now computable in polynomial time as a semidefinite program. We also provide computationally more efficient relaxations based on spectral information divergences from quantum information theory. For all of the tasks above, beyond proposing new relaxations, we derive tractable convex optimization algorithms, and we present illustrations on multivariate trigonometric polynomials and functions on the Boolean hypercube.
Optimal and Robust Category-Level Perception: Object Pose and Shape Estimation From 2-D and 3-D Semantic Keypoints

In this article, we consider a <italic>category-level perception</italic> problem, where one is given 2-D or 3-D sensor data picturing an object of a given category (e.g., a car) and has to reconstruct the 3-D pose and shape of the object despite intraclass variability (i.e., different car models have different shapes). We consider an <italic>active shape model</italic>, where—for an object category—we are given a library of potential computer-aided design models describing objects in that category, and we adopt a standard formulation where pose and shape are estimated from 2-D or 3-D keypoints via nonconvex optimization. Our first contribution is to develop <monospace>PACE3D</monospace><inline-formula><tex-math notation="LaTeX">$^{\star }$</tex-math></inline-formula> and <monospace>PACE2D</monospace><inline-formula><tex-math notation="LaTeX">$^{\star }$</tex-math></inline-formula>, the first <italic>certifiably optimal</italic> solvers for pose and shape estimation using 3-D and 2-D keypoints, respectively. Both the solvers rely on the design of tight (i.e., exact) semidefinite relaxations. Our second contribution is to develop outlier-robust versions of both the solvers, named <monospace>PACE3D</monospace># and <monospace>PACE2D</monospace>#. Toward this goal, we propose ROBIN(<italic>Reject Outliers Based on INvariants</italic>), a general graph-theoretic framework to prune outliers, which uses <italic>compatibility hypergraphs</italic> to model measurements' compatibility. We show that in category-level perception problems, these hypergraphs can be built from the winding orders of the keypoints (in 2-D) or their convex hulls (in 3-D), and many outliers can be filtered out via maximum hyperclique computation. The last contribution is an extensive experimental evaluation. Besides providing an ablation study on simulated datasets and on the <monospace>PASCAL3D</monospace>+ dataset, we combine our solver with a deep keypoint detector and show that <monospace>PACE3D</monospace># improves over the state of the art in vehicle pose estimation in the <monospace>ApolloScape</monospace> datasets, and its runtime is compatible with practical applications. We release our code at <uri>https://github.com/MIT-SPARK/PACE</uri>.
Nonwatertight Mesh Reconstruction

Reconstructing 3D non-watertight mesh from an unoriented point cloud is an unexplored area in computer vision and computer graphics. In this project, we tried to tackle this problem by extending the learning-based watertight mesh reconstruction pipeline presented in the paper 'Shape as Points'. The core of our approach is to cast the problem as a semantic segmentation problem that identifies the region in the 3D volume where the mesh surface lies and extracts the surfaces from the detected regions. Our approach achieves compelling results compared to the baseline techniques.