Miracle-dz/UniFAR-FTUs huggingface dataset:
Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers

Lip reading has witnessed unparalleled development in recent years thanks to deep learning and the availability of large-scale datasets. Despite the encouraging results achieved, the performance of lip reading, unfortunately, remains inferior to the one of its counterpart speech recognition, due to the ambiguous nature of its actuations that makes it challenging to extract discriminant features from the lip movement videos. In this paper, we propose a new method, termed as Lip by Speech (LIBS), of which the goal is to strengthen lip reading by learning from speech recognizers. The rationale behind our approach is that the features extracted from speech recognizers may provide complementary and discriminant clues, which are formidable to be obtained from the subtle movements of the lips, and consequently facilitate the training of lip readers. This is achieved, specifically, by distilling multi-granularity knowledge from speech recognizers to lip readers. To conduct this cross-modal knowledge distillation, we utilize an efficacious alignment scheme to handle the inconsistent lengths of the audios and videos, as well as an innovative filtering strategy to refine the speech recognizer's prediction. The proposed method achieves the new state-of-the-art performance on the CMLR and LRS2 datasets, outperforming the baseline by a margin of 7.66% and 2.75% in character error rate, respectively.
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

We study the problem of efficient semantic segmentation for large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Extensive experiments show that our RandLA-Net can process 1 million points in a single pass with up to 200x faster than existing approaches. Moreover, our RandLA-Net clearly surpasses state-of-the-art approaches for semantic segmentation on two large-scale benchmarks Semantic3D and SemanticKITTI.
Gravitational anomalies in nAdS2/nCFT1

We revisit the holographic description of the near horizon geometry of the BTZ black hole in AdS3 gravity, with a gravitational Chern–Simons term included. After a dimensional reduction of the three dimensional theory, we use the framework of nAdS2/nCFT1 to describe the near horizon physics. This setup allows us to contrast the role of the gravitational and conformal anomaly inherited from AdS3/CFT2 in the symmetry breaking mechanism of nAdS2/nCFT1. Our results display how boundary conditions in the 3D spacetime, combined with the gravitational anomaly, affect the holographic description of the near horizon of the black hole relative to the physics near the AdS3 boundary.
SRG: Snippet Relatedness-Based Temporal Action Proposal Generator

Recent temporal action proposal generation approaches have suggested integrating segment- and snippet score-based methodologies to produce proposals with high recall and accurate boundaries. In this paper, different from such a hybrid strategy, we focus on the potential of the snippet score-based approach. Specifically, we propose a new snippet score-based method, named Snippet Relatedness-based Generator (SRG), with a novel concept of “snippet relatedness”. Snippet relatedness represents which snippets are related to a specific action instance. To effectively learn this snippet relatedness, we present “pyramid non-local operations” for locally and globally capturing long-range dependencies among snippets. By employing these components, SRG first produces a 2D relatedness score map that enables the generation of various temporal intervals reliably covering most action instances with high overlap. Then, SRG evaluates the action confidence scores of these temporal intervals and refines their boundaries to obtain temporal action proposals. On THUMOS-14 and ActivityNet-1.3 datasets, SRG outperforms state-of-the-art methods for temporal action proposal generation. Furthermore, compared to competing proposal generators, SRG leads to significant improvements in temporal action detection.
Fuzzy Deterministic Top-down Tree Automata

In this paper we introduce and study fuzzy deterministic top-down (DT) tree automata over a lattice L. The L-fuzzy tree languages recognized by these automata are said to be DT-recognizable, and they form a proper subfamily $DRec_L$ of the family of $Rec_L$ of all regular L-fuzzy tree languages. We prove a Pumping Lemma for $DRec_L$ from which several decidability results follow. The closure properties of $DRec_L$ under various operations are established. We also characterize DT-recognizability in terms of L-fuzzy path languages, and prove that the path closure of any regular L-fuzzy tree language is DT-recognizable, and that it is decidable whether a regular L-fuzzy tree language is DT-recognizable. In most of the paper, L is just any nontrivial bounded lattice, but sometimes it is assumed to be distributive or even a bounded chain.
Neural Graph Matching Network: Learning Lawler’s Quadratic Assignment Problem With Extension to Hypergraph and Multiple-Graph Matching

Graph matching involves combinatorial optimization based on edge-to-edge affinity matrix, which can be generally formulated as Lawler's quadratic assignment problem (QAP). This paper presents a QAP network directly learning with the affinity matrix (equivalently the association graph) whereby the matching problem is translated into a constrained vertex classification task. The association graph is learned by an embedding network for vertex classification, followed by Sinkhorn normalization and a cross-entropy loss for end-to-end learning. We further improve the embedding model on association graph by introducing Sinkhorn based matching-aware constraint, as well as dummy nodes to deal with unequal sizes of graphs. To our best knowledge, this is one of the first network to directly learn with the general Lawler's QAP. In contrast, recent deep matching methods focus on the learning of node/edge features in two graphs respectively. We also show how to extend our network to hypergraph matching, and matching of multiple graphs. Experimental results on both synthetic graphs and real-world images show its effectiveness. For pure QAP tasks on synthetic data and QAPLIB benchmark, our method can perform competitively and even surpass state-of-the-art graph matching and QAP solvers with notable less time cost. We provide a project homepage at http://thinklab.sjtu. edu.cn/project/NGM/index.html.
In a physics curriculum only introductory physics course grades show gender differences but do not predict future course performance for physics majors

Analysis of institutional data for physics majors showing predictive relationships between required mathematics and physics courses in various years is important for contemplating how the courses build on each other and whether there is need to make changes to the curriculum for the majors to strengthen these relationships. We use 15 years of institutional data at a large research university to investigate how introductory physics and mathematics courses predict male and female physics majors' performance on required advanced physics and mathematics courses. We used Structure Equation Modeling (SEM) to investigate these predictive relationships and find that among introductory and advanced physics and mathematics courses, there are gender differences in performance in favor of male students only in the introductory physics courses after controlling for high school GPA. We found that a measurement invariance fully holds in a multi-group SEM by gender, so it was possible to carry out analysis with gender mediated by introductory physics and high school GPA. Moreover, we find that these introductory physics courses that have gender differences do not predict performance in advanced physics courses. Also, introductory mathematics courses predict performance in advanced mathematics courses which in turn predict performance in advanced physics courses. Furthermore, apart from the introductory physics courses that do not predict performance in future physics courses, there is a strong predictive relationship between the sophomore, junior and senior level physics courses.
MULTIFIDELITY ESTIMATORS FOR CORONARY CIRCULATION MODELS UNDER CLINICALLY INFORMED DATA UNCERTAINTY

Numerical models are increasingly used for non-invasive diagnosis and treatment planning in coronary artery disease, where service-based technologies have proven successful in identifying hemodynamically significant and hence potentially dangerous vascular anomalies. Despite recent progress towards clinical adoption, many results in the field are still based on a deterministic characterization of blood flow, with no quantitative assessment of the variability of simulation outputs due to uncertainty from multiple sources. In this study, we focus on parameters that are essential to construct accurate patient-specific representations of the coronary circulation, such as aortic pressure waveform, intramyocardial pressure and quantify how their uncertainty affects clinically relevant model outputs. We construct a deformable model of the left coronary artery subject to a prescribed inlet pressure and with open-loop outlet boundary conditions, treating fluid-structure interaction through an Arbitrary-Lagrangian-Eulerian frame of reference. Random input uncertainty is estimated directly from repeated clinical measurements from intra-coronary catheterization and complemented by literature data. We also achieve significant computational cost reductions in uncertainty propagation thanks to multifidelity Monte Carlo estimators of the outputs of interest, leveraging the ability to generate, at practically no cost, one- and zero-dimensional low-fidelity representations of left coronary artery flow, with appropriate boundary conditions. The results demonstrate how the use of multi-fidelity control variate estimators leads to significant reductions in variance and accuracy improvements with respect to traditional Monte-Carlo.
Prediction of Horizontal Data Partitioning Through Query Execution Cost Estimation

The excessively increased volume of data in modern data management systems demands an improved system performance, frequently provided by data distribution, system scalability and performance optimization techniques. Optimized horizontal data partitioning has a significant influence of distributed data management systems. An optimally partitioned schema found in the early phase of logical database design without loading of real data in the system and its adaptation to changes of business environment are very important for a successful implementation, system scalability and performance improvement. In this paper we present a novel approach for finding an optimal horizontally partitioned schema that manifests a minimal total execution cost of a given database workload. Our approach is based on a formal model that enables abstraction of the predicates in the workload queries, and are subsequently used to define all relational fragments. This approach has predictive features acquired by simulation of horizontal partitioning, without loading any data into the partitions, but instead, altering the statistics in the database catalogs. We define an optimization problem and employ a genetic algorithm (GA) to find an approximately optimal horizontally partitioned schema. The solutions to the optimization problem are evaluated using PostgreSQL's query optimizer. The initial experimental evaluation of our approach confirms its efficiency and correctness, and the numbers imply that the approach is effective in reducing the workload execution cost.
Polynomial Trace Identities in SL(2,C), Quaternion Algebras, and Two-generator Kleinian Groups

We study certain polynomial trace identities in the group $SL(2,\IC)$ and their application in the theory of discrete groups. We obtain canonical representations for two generator groups in §4 and then in §5 we give a new proof for Gehring and Martin's polynomial trace identities for good words, and extend that result to a larger class which is also closed under a semigroup operation inducing polynomial composition. This new approach is through the use of quaternion algebras over indefinites and an associated group of units. We obtain structure theorems for these quaternion algebras which appear to be of independent interest in §8. Using these quaternion algebras and their units, we consider their relation to discrete subgroups of $SL(2,\IC)$ giving necessary and sufficient criteria for discreteness, and another for arithmeticity §9. We then show that for the groups $\IZ_p*\IZ_2$, the complement of the closure of roots of the good word polynomials is precisely the moduli space of geometrically finite discrete and faithful representations a result we show holds in greater generality in \S12.
A Comparison of Partitioning Strategies in AC Optimal Power Flow

The highly non-convex AC optimal power flow problem is known to scale very poorly with respect to the number of lines and buses. To achieve improved computational speed and scalability, we apply a distributed optimization algorithm, the so-called Augmented Lagrangian based Alternating Direction Inexact Newton (ALADIN) method. However, the question of how to optimally partition a power grid for use in distributed optimization remains open in the literature. In the present paper, we compare four partitioning strategies using the standard IEEE 9, 14, 30, 39, 57, 118, 300 bus models as the performance benchmark. To this end, we apply the graph partitioners KaFFPa and METIS as well as two spectral clustering methods to the aforementioned bus models. For larger grids, KaFFPa yields the best results on average.
Path Integral Renormalization of Flow through Random Porous Media

The path integral for Darcy's law with a stochastic conductivity, which characterizes flow through random porous media, is used as a basis for Wilson renormalization-group (RG) calculations in momentum space. A coarse graining procedure is implemented by integrating over infinitesimal shells of large momenta corresponding to the elimination of the small scale modes of the theory. The resulting one-loop $\beta$-functions are solved exactly to obtain an effective conductivity in a coarse grained theory over successively larger length scales. We first carry out a calculation with uncorrelated Gaussian conductivity fluctuations to illustrate the RG procedure before considering the effect of a finite correlation length of conductivity fluctuations. We conclude by discussing applications and extensions of our calculations, including comparisons with the numerical evaluation of path integrals, non-Gaussian fluctuations, and multiphase flow, for which the path integral formulation should prove particularly useful.
Multi-source Distilling Domain Adaptation

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA.
The spatiotemporal tau statistic: a review

Introduction The tau statistic is a recent second-order correlation function that can assess the magnitude and range of global spatiotemporal clustering from epidemiological data containing geolocations of individual cases and, usually, disease onset times. This is the first review of its use, and the aspects of its computation and presentation that could affect inferences drawn and bias estimates of the statistic. Methods Using Google Scholar we searched papers or preprints that cited the papers that first defined/reformed the statistic. We tabulated their key characteristics to understand the statistic's development since 2012. Results Only half of the 16 studies found were considered to be using true tau statistics, but their inclusion in the review still provided important insights into their analysis motivations. All papers that used graphical hypothesis testing and parameter estimation used incorrect methods. There is a lack of clarity over how to choose the time-relatedness interval to relate cases and the distance band set, that are both required to calculate the statistic. Some studies demonstrated nuanced applications of the tau statistic in settings with unusual data or time relation variables, which enriched understanding of its possibilities. A gap was noticed in the estimators available to account for variable person-time at risk. Discussion Our review comprehensively covers current uses of the tau statistic for descriptive analysis, graphical hypothesis testing, and parameter estimation of spatiotemporal clustering. We also define a new estimator of the tau statistic for disease rates. For the tau statistic there are still open questions on its implementation which we hope this review inspires others to research.
LogPlayer: Fault-tolerant Exactly-once Delivery using gRPC Asynchronous Streaming

In this paper, we present the design of our LogPlayer that is a component responsible for fault-tolerant delivery of transactional mutations recorded on a WAL to the backend storage shards. LogPlayer relies on gRPC for asynchronous streaming. However, the design provided in this paper can be used with other asynchronous streaming platforms. We model check the correctness of LogPlayer by TLA+. In particular, our TLA+ specification shows that LogPlayer guarantees in-order exactly-once delivery of WAL entries to the storage shards, even in the presence of shards or LogPlayer failures. Our experiments show LogPlayer is capable of efficient delivery with sub-millisecond latency, and it is significantly more efficient than Apache Kafka for designing a WAL system with exactly-once guarantee.
Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous Multi-Lane Driving

Autonomous driving promises to transform road transport. Multi-vehicle and multi-lane scenarios, however, present unique challenges due to constrained navigation and unpredictable vehicle interactions. Learning-based methods---such as deep reinforcement learning---are emerging as a promising approach to automatically design intelligent driving policies that can cope with these challenges. Yet, the process of safely learning multi-vehicle driving behaviours is hard: while collisions---and their near-avoidance---are essential to the learning process, directly executing immature policies on autonomous vehicles raises considerable safety concerns. In this article, we present a safe and efficient framework that enables the learning of driving policies for autonomous vehicles operating in a shared workspace, where the absence of collisions cannot be guaranteed. Key to our learning procedure is a sim2real approach that uses real-world online policy adaptation in a mixed-reality setup, where other vehicles and static obstacles exist in the virtual domain. This allows us to perform safe learning by simulating (and learning from) collisions between the learning agent(s) and other objects in virtual reality. Our results demonstrate that, after only a few runs in mixed-reality, collisions are significantly reduced.
Impressive computational acceleration by using machine learning for 2-dimensional super-lubricant materials discovery

The screening of novel materials is an important topic in the field of materials science. Although traditional computational modeling, especially first-principles approaches, is a very useful and accurate tool to predict the properties of novel materials, it still demands extensive and expensive state-of-the-art computational resources. Additionally, they can be often extremely time consuming. We describe a time and resource-efficient machine learning approach to create a large dataset of structural properties of van der Waals layered structures. In particular, we focus on the interlayer energy and the elastic constant of layered materials composed of two different 2-dimensional (2D) structures, that are important for novel solid lubricant and super-lubricant materials. We show that machine learning models can recapitulate results of computationally expansive approaches (i.e. density functional theory) with high accuracy.
Sewing states of quantum field theory

Consider an n-partite system and denote by ω the local density matrix at site Ai. We say a pure n-partite state |Ω〉 sews ω together if it reduces to ω on Ai for all i. In finite quantum systems, density matrices can be sewn together only if their eigenvalues satisfy polygon inequalities. We show that in quantum field theory there are no constraints on sewing local states. The reason is that rotating ω by a unitary Ui in Ai we come arbitrarily close to any other ψ . We construct explicit unitaries that sew local states.