Miracle-dz/UniFAR-FTUs huggingface dataset:
Beyond Exact Gradients: Convergence of Stochastic Soft-Max Policy Gradient Methods with Entropy Regularization

Entropy regularization is an efficient technique for encouraging exploration and preventing a premature convergence of (vanilla) policy gradient methods in reinforcement learning (RL). However, the theoretical understanding of entropy regularized RL algorithms has been limited. In this paper, we revisit the classical entropy regularized policy gradient methods with the soft-max policy parametrization, whose convergence has so far only been established assuming access to exact gradient oracles. To go beyond this scenario, we propose the first set of (nearly) unbiased stochastic policy gradient estimators with trajectory-level entropy regularization, with one being an unbiased visitation measure-based estimator and the other one being a nearly unbiased yet more practical trajectory-based estimator. We prove that although the estimators themselves are unbounded in general due to the additional logarithmic policy rewards introduced by the entropy term, the variances are uniformly bounded. We then propose a two-phase stochastic policy gradient (PG) algorithm that uses a large batch size in the first phase to overcome the challenge of the stochastic approximation due to the non-coercive landscape, and uses a small batch size in the second phase by leveraging the curvature information around the optimal policy. We establish a global optimality convergence result and a sample complexity of $\widetilde{\mathcal{O}}(\frac{1}{\epsilon^2})$ for the proposed algorithm. Our result is the first global convergence and sample complexity results for the stochastic entropy-regularized vanilla PG method.
Label-Descriptive Patterns and their Application to Characterizing Classification Errors

State-of-the-art deep learning methods achieve human-like performance on many tasks, but make errors nevertheless. Characterizing these errors in easily interpretable terms gives insight into whether a model is prone to making systematic errors, but also gives a way to act and improve the model. In this paper we propose a method that allows us to do so for arbitrary classiﬁers by mining a small set of patterns that together succinctly describe the input data that is partitioned according to correctness of prediction. We show this is an instance of the more general label description problem, which we formulate in terms of the Minimum Description Length principle. To discover good pattern sets we propose the eﬃcient and hyperparameter-free Premise algorithm, which through an extensive set of experiments we show on both synthetic and real-world data performs very well in practice; unlike existing solutions it ably recovers ground truth patterns, even on highly imbalanced data over many unique items, or where patterns are only weakly associated to labels. Through two real-world case studies we conﬁrm that Premise gives clear and actionable insight into the systematic errors made by modern NLP classiﬁers.
Generalized quasi-Keplerian solution for eccentric, non-spinning compact binaries at 4PN order and the associated IMR waveform

(Dated: We derive fourth post-Newtonian (4PN) contributions to the Keplerian-type parametric solution associated with the conservative dynamics of eccentric, non-spinning compact binaries. The solution has been computed while ignoring certain zero-average, oscillatory terms arising due to 4PN tail eﬀects. We provide explicit expressions for the parametric solution and various orbital elements in terms of the conserved energy, angular momentum and symmetric mass ratio. Canonical perturbation theory (along with the technique of Pad´e approximant) is used to incorporate the 4PN nonlocal-in-time tail eﬀects within the action-angles framework. We then employ the resulting solution to obtain an updated inspiral-merger-ringdown (IMR) waveform that models the coalescence of non-spinning, moderately eccentric black hole binaries, inﬂuenced by Ref. [I. Hinder et al. , Phys. Rev. D 98, 044015 (2018)]. Our updated waveform is expected to be valid over similar parameter range as the above reference. We also present a related waveform which makes use of only the post-Newtonian equations and thus is valid only for the inspiral stage. This waveform is expected to work for a much larger range of eccentricity ( e t < ∼ 0 . 85) than our full IMR waveform (which assumes circularization of the binaries close to merger). We ﬁnally pursue preliminary data analysis studies to probe the importance of including the 4PN contributions to the binary dynamics while constructing gravitational waveform templates for eccentric mergers.
Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry

This work studies learning from a synergy process of 3D Morphable Models (3DMM) and 3D facial landmarks to predict complete 3D facial geometry, including 3D alignment, face orientation, and 3D face modeling. Our synergy process leverages a representation cycle for 3DMM parameters and 3D landmarks. 3D landmarks can be extracted and refined from face meshes built by 3DMM parameters. We next reverse the representation direction and show that predicting 3DMM parameters from sparse 3D landmarks improves the information flow. Together we create a synergy process that utilizes the relation between 3D landmarks and 3DMM parameters, and they collaboratively contribute to better performance. We extensively validate our contribution on full tasks of facial geometry prediction and show our superior and robust performance on these tasks for various scenarios. Particularly, we adopt only simple and widely-used network operations to attain fast and accurate facial geometry prediction. Codes and data: https: //choyingw.github. io/works/SynergyNet/.
Learning to Learn Graph Topologies

Learning a graph topology to reveal the underlying relationship between data entities plays an important role in various machine learning and data analysis tasks. Under the assumption that structured data vary smoothly over a graph, the problem can be formulated as a regularised convex optimisation over a positive semidefinite cone and solved by iterative algorithms. Classic methods require an explicit convex function to reflect generic topological priors, e.g. the $\ell_1$ penalty for enforcing sparsity, which limits the flexibility and expressiveness in learning rich topological structures. We propose to learn a mapping from node data to the graph structure based on the idea of learning to optimise (L2O). Specifically, our model first unrolls an iterative primal-dual splitting algorithm into a neural network. The key structural proximal projection is replaced with a variational autoencoder that refines the estimated graph with enhanced topological properties. The model is trained in an end-to-end fashion with pairs of node data and graph samples. Experiments on both synthetic and real-world data demonstrate that our model is more efficient than classic iterative algorithms in learning a graph with specific topological properties.
Non-local gravity in bouncing cosmology scenarios

In this work, we analyzed the improved Deser-Woodard non-local gravity over the background of five different bouncing cosmologies, whose premise is avoid the initial singular state of the universe. We developed the numerical solution for the non-local distortion function, which encompass the modifications to the Einstein-Hilbert action, using the reconstruction procedure and we have found that they have a viable cosmological solution. Afterwards, we discussed the physical aspects and outcomes of the evolution of the distortion function throughout the bouncing point for these models, specifically: the symmetric bounce, oscillatory bounce, the matter bounce, finite time singularity model, and the pre-inflationary asymmetrical bounce.
Speed Limits for Macroscopic Transitions

Speed of state transitions in macroscopic systems is a crucial concept for foundations of nonequilibrium statistical mechanics as well as various applications in quantum technology represented by optimal quantum control. While extensive studies have made eﬀorts to obtain rigorous constraints on dynamical processes since Mandelstam and Tamm, speed limits that provide tight bounds for macroscopic transitions have remained elusive. Here, by employing the local conservation law of probability, the fundamental principle in physics, we develop a general framework for deriving qualitatively tighter speed limits for macroscopic systems than many conventional ones. We show for the ﬁrst time that the speed of the expectation value of an observable deﬁned on an arbitrary graph, which can describe general many-body systems, is bounded by the “gradient” of the observable, in contrast with conventional speed limits depending on the entire range of the observable. This framework enables us to derive novel quantum speed limits for macroscopic unitary dynamics. Unlike previous bounds, the speed limit decreases when the expectation value of the transition Hamiltonian increases; this intuitively describes a new tradeoﬀ relation between time and quantum phase diﬀerence. Our bound is dependent on instantaneous quantum states and thus can achieve the equality condition, which is conceptually distinct from the Lieb-Robinson bound. We also ﬁnd that, beyond expectation values of macroscopic observables, the speed of macroscopic quantum coherence can be bounded from above by our general approach. The newly obtained bounds are veriﬁed in transport phenomena in particle systems and nonequilibrium dynamics in many-body spin systems. We also demonstrate that our strategy can be applied for ﬁnding new speed limits for macroscopic transitions in stochastic systems, including quantum ones, where the bounds are expressed by the entropy production rate. Our work elucidates novel speed limits on the basis of local conservation law, providing fundamental limits to various types of nonequilibrium quantum macroscopic phenomena.
Coarse-graining and reconstruction for Markov matrices

We present a coarse-graining (or model order reduction) procedure for stochastic matrices by clustering. The method is consistent with the natural structure of Markov theory, preserving positivity and mass, and does not rely on any tools from Hilbert space theory. The reconstruction is provided by a generalized Penrose-Moore inverse of the coarsegraining operator incorporating the inhomogeneous invariant measure of the Markov matrix. As we show, the method provides coarse-graining and reconstruction also on the level of tensor spaces, which is consistent with the notion of an incidence matrix and quotient graphs, and, moreover, allows to coarse-grain and reconstruct fluxes. Furthermore, we investigate the connection with functional inequalities and Poincaré-type constants.
Talking Head Generation with Audio and Speech Related Facial Action Units

The task of talking head generation is to synthesize a lip synchronized talking head video by inputting an arbitrary face image and audio clips. Most existing methods ignore the local driving information of the mouth muscles. In this paper, we propose a novel recurrent generative network that uses both audio and speech-related facial action units (AUs) as the driving information. AU information related to the mouth can guide the movement of the mouth more accurately. Since speech is highly correlated with speech-related AUs, we propose an Audio-to-AU module in our system to predict the speech-related AU information from speech. In addition, we use AU classifier to ensure that the generated images contain correct AU information. Frame discriminator is also constructed for adversarial training to improve the realism of the generated face. We verify the effectiveness of our model on the GRID dataset and TCD-TIMIT dataset. We also conduct an ablation study to verify the contribution of each component in our model. Quantitative and qualitative experiments demonstrate that our method outperforms existing methods in both image quality and lip-sync accuracy.
Flexible Accuracy for Differential Privacy

Differential Privacy (DP) has become a gold standard in privacy-preserving data analysis. While it provides one of the most rigorous notions of privacy, there are many settings where its applicability is limited. Our main contribution is in augmenting differential privacy with {\em Flexible Accuracy}, which allows small distortions in the input (e.g., dropping outliers) before measuring accuracy of the output, allowing one to extend DP mechanisms to high-sensitivity functions. We present mechanisms that can help in achieving this notion for functions that had no meaningful differentially private mechanisms previously. In particular, we illustrate an application to differentially private histograms, which in turn yields mechanisms for revealing the support of a dataset or the extremal values in the data. Analyses of our constructions exploit new versatile composition theorems that facilitate modular design. All the above extensions use our new definitional framework, which is in terms of"lossy Wasserstein distance"-- a 2-parameter error measure for distributions. This may be of independent interest.
Locally Differentially Private Reinforcement Learning for Linear Mixture Markov Decision Processes

Reinforcement learning (RL) algorithms can be used to provide personalized services, which rely on users' private and sensitive data. To protect the users' privacy, privacy-preserving RL algorithms are in demand. In this paper, we study RL with linear function approximation and local differential privacy (LDP) guarantees. We propose a novel $(\varepsilon, \delta)$-LDP algorithm for learning a class of Markov decision processes (MDPs) dubbed linear mixture MDPs, and obtains an $\tilde{\mathcal{O}}( d^{5/4}H^{7/4}T^{3/4}\left(\log(1/\delta)\right)^{1/4}\sqrt{1/\varepsilon})$ regret, where $d$ is the dimension of feature mapping, $H$ is the length of the planning horizon, and $T$ is the number of interactions with the environment. We also prove a lower bound $\Omega(dH\sqrt{T}/\left(e^{\varepsilon}(e^{\varepsilon}-1)\right))$ for learning linear mixture MDPs under $\varepsilon$-LDP constraint. Experiments on synthetic datasets verify the effectiveness of our algorithm. To the best of our knowledge, this is the first provable privacy-preserving RL algorithm with linear function approximation.
Latent reweighting, an almost free improvement for GANs

Standard formulations of GANs, where a continuous function deforms a connected latent space, have been shown to be misspecified when fitting different classes of images. In particular, the generator will necessarily sample some low-quality images in between the classes. Rather than modifying the architecture, a line of works aims at improving the sampling quality from pre-trained generators at the expense of increased computational cost. Building on this, we introduce an additional network to predict latent importance weights and two associated sampling methods to avoid the poorest samples. This idea has several advantages: 1) it provides a way to inject disconnectedness into any GAN architecture, 2) since the rejection happens in the latent space, it avoids going through both the generator and the discriminator, saving computation time, 3) this importance weights formulation provides a principled way to reduce the Wasserstein’s distance to the target distribution. We demonstrate the effectiveness of our method on several datasets, both synthetic and high-dimensional.
Entity Relation Extraction as Dependency Parsing in Visually Rich Documents

Previous works on key information extraction from visually rich documents (VRDs) mainly focus on labeling the text within each bounding box (i.e.,semantic entity), while the relations in-between are largely unexplored. In this paper, we adapt the popular dependency parsing model, the biaffine parser, to this entity relation extraction task. Being different from the original dependency parsing model which recognizes dependency relations between words, we identify relations between groups of words with layout information instead. We have compared different representations of the semantic entity, different VRD encoders, and different relation decoders. For the model training, we explore multi-task learning to combine entity labeling and relation extraction tasks; and for the evaluation, we conduct experiments on different datasets with filtering and augmentation. The results demonstrate that our proposed model achieves 65.96% F1 score on the FUNSD dataset. As for the real-world application, our model has been applied to the in-house customs data, achieving reliable performance in the production setting.
Covariant Guiding Laws for Fields

After reviewing what is known about the passage from the classical Hamilton--Jacobi formulation of non-relativistic point-particle dynamics to the non-relativistic quantum dynamics of point particles whose motion is guided by a wave function that satisfies Schr\"odinger's or Pauli's equation, we study the analogous question for the Lorentz-covariant dynamics of fields on spacelike slices of spacetime. We establish a relationship, between the DeDonder--Weyl--Christodoulou formulation of covariant Hamilton--Jacobi equations for the classical field evolution, and the Lorentz-covariant Dirac-type wave equation proposed by Kanatchikov amended by our proposed guiding equation for such fields. We show that Kanatchikov's equation is well-posed and generally solvable, and we establish the correspondence between plane-wave solutions of Kanatchikov's equation and solutions of the covariant Hamilton--Jacobi equations of DeDonder--Weyl--Christodoulou. We propose a covariant guiding law for the temporal evolution of fields defined on constant time slices of spacetime, and show that it yields, at each spacetime point, the existence of a finite measure on the space of field values at that point that is equivariant with respect to the flow induced by the solution of Kanatchikov's equation that is guiding the actual field, so long as it is a plane-wave solution. We show that our guiding law is local in the sense of Einstein's special relativity, and therefore it cannot be used to analyze Bell-type experiments. We conclude by suggesting directions to be explored in future research.
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents

Progress in continual reinforcement learning has been limited due to several barriers to entry: missing code, high compute requirements, and a lack of suitable benchmarks. In this work, we present CORA, a platform for Continual Reinforcement Learning Agents that provides benchmarks, baselines, and metrics in a single code package. The benchmarks we provide are designed to evaluate different aspects of the continual RL challenge, such as catastrophic forgetting, plasticity, ability to generalize, and sample-efficient learning. Three of the benchmarks utilize video game environments (Atari, Procgen, NetHack). The fourth benchmark, CHORES, consists of four different task sequences in a visually realistic home simulator, drawn from a diverse set of task and scene parameters. To compare continual RL methods on these benchmarks, we prepare three metrics in CORA: Continual Evaluation, Isolated Forgetting, and Zero-Shot Forward Transfer. Finally, CORA includes a set of performant, open-source baselines of existing algorithms for researchers to use and expand on. We release CORA and hope that the continual RL community can benefit from our contributions, to accelerate the development of new continual RL algorithms.
Extensive Deep Temporal Point Process

—Temporal point process as the stochastic process on continuous domain of time is usually used to model the asynchronous event sequence featuring with occurence timestamps. With the rise of deep learning, due to the strong expressivity of deep neural networks, they are emerging as a promising choice for capturing the patterns in asynchronous sequences, in the setting of temporal point process. In this paper, we ﬁrst review recent research emphasis and difﬁculties in modeling asynchronous event sequences with deep temporal point process, which can be concluded into four ﬁelds: encoding of history sequence, formulation of conditional intensity function, relational discovery of events and learning approaches for optimization. We introduce most of recently proposed models by dismantling them as the four parts, and conduct experiments by remodularizing the ﬁrst three parts with the same learning strategy for a fair empirical evaluation. Besides, we extend the history encoders and conditional intensity function family, and propose a Granger causality discovery framework for exploiting the relations among multi-types of events. Discrete graph structure learning in the framework of Variational Inference is employed to reveal latent structures of Granger causality graph, and further experiments shows the proposed framework with learned latent graph can both capture the relations and achieve an improved ﬁtting and predicting performance.
Inductive Biases and Variable Creation in Self-Attention Mechanisms

Self-attention, an architectural motif designed to model long-range interactions in sequential data, has driven numerous recent breakthroughs in natural language processing and beyond. This work provides a theoretical analysis of the inductive biases of self-attention modules. Our focus is to rigorously establish which functions and long-range dependencies self-attention blocks prefer to represent. Our main result shows that bounded-norm Transformer networks"create sparse variables": a single self-attention head can represent a sparse function of the input sequence, with sample complexity scaling only logarithmically with the context length. To support our analysis, we present synthetic experiments to probe the sample complexity of learning sparse Boolean functions with Transformers.
BAMLD: Bayesian Active Meta-Learning by Disagreement

Data-eﬃcient learning algorithms are essential in many practical applications for which data collection and labeling is expensive or infeasible, e.g., for autonomous cars. To address this problem, meta-learning infers an inductive bias from a set of meta-training tasks in order to learn new, but related, task using a small number of samples. Most studies assume the meta-learner to have access to labeled data sets from a large number of tasks. In practice, one may have available only unlabeled data sets from the tasks, requiring a costly labeling procedure to be carried out before use in standard meta-learning schemes. To decrease the number of labeling requests for meta-training tasks, this paper introduces an information-theoretic active task selection mechanism which quan-tiﬁes the epistemic uncertainty via disagreements among the predictions obtained under diﬀerent inductive biases. We detail an in-stantiation for nonparametric methods based on Gaussian Process Regression, and report its empirical performance results that compare favourably against existing heuristic acquisition mechanisms.