Miracle-dz/UniFAR-FTUs huggingface dataset:
Exploring the climate system through manifold learning

The threat of global warming and the demand for confident climate predictions pose a formidable challenge. The climate system is multiscale, high-dimensional and nonlinear. However, spatiotemporal recurrences of the system hint to the presence of low-dimensional manifolds containing the high-dimensional climate trajectory. Reproducing the geometrical and topological properties of the low-dimensional attractor is an important constraint for models to generate meaningful probabilistic climate projections. Here we propose a framework to characterize and compare the climate attractor across observations and models. This framework is showcased in the tropical Pacific ocean using a reanalysis as observational proxy and two state-of-the-art models. The analysis spans four variables over the periods 1979-2019 and 2060-2100. At each time t, the system can be uniquely described by a state space vector parameterized by N variables and their spatial variability. The dynamics is confined on a manifold with dimension lower than the full state space that can be characterized through manifold learning algorithms, both linear and nonlinear. The manifold learning step allows for quantifying nonlinear contributions as well as the relative role of each variable in the system’s dynamics. Geometric properties of the high-dimensional attractor are further quantified through the local dimension and persistence metrics, recently proposed in the climate literature. The metrics quantify the local geometry of the climate attractor and its local stability. Both models underestimate the average dimension and overestimate persistence compared to observations. These biases limit climate predictability and are nearly identical among the chosen models during the historical period while diverging in the near future under the warming scenario considered.
Injecting Planning-Awareness into Prediction and Detection Evaluation

Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios entailing human-robot interaction such as autonomous driving. Due to the importance of these components, there has been a significant amount of interest and research in perception and trajectory forecasting, resulting in a wide variety of approaches. Common to most works, however, is the use of the same few accuracy-based evaluation metrics, e.g., intersection-over-union, displacement error, log-likelihood, etc. While these metrics are informative, they are task-agnostic and outputs that are evaluated as equal can lead to vastly different outcomes in downstream planning and decision making. In this work, we take a step back and critically assess current evaluation metrics, proposing task-aware metrics as a better measure of performance in systems where they are deployed. Experiments on an illustrative simulation as well as real-world autonomous driving data validate that our proposed task-aware metrics are able to account for outcome asymmetry and provide a better estimate of a model’s closed-loop performance.
Chemical constituents of essential oils from the leaves of Tithonia diversifolia, Houttuynia cordata and Asarum glabrum grown in Vietnam

The essential oil constituents of three medicinal plants grown in Vietnam were analyzed by gas chromatography-flame ionization detector (GC-FID) and gas chromatography-mass spectrometry (GCMS) techniques. The main constituents of Tithonia diversifolia (Hemsl.) A. Gray (Asteraceae) were pinene (30.7%) along with (E,E)--farnesene (6.1%) and -caryophyllene (5.1%). Houttuynia cordata Thunb. (Saururaceae) gave oil whose major components were -myrcene (30.8%), 2-undecanone (19.7%) and (Z)--ocimene (10.2%). Asarum glabrum Merr. (Aristolochiaceae) consists mainly of safrole (46.6%) and apiole (17.0%).
From Contraction Theory to Fixed Point Algorithms on Riemannian and Non-Euclidean Spaces

The design of fixed point algorithms is at the heart of monotone operator theory, convex analysis, and of many modern optimization problems arising in machine learning and control. This tutorial reviews recent advances in understanding the relationship between Demidovich conditions, one-sided Lipschitz conditions, and contractivity theorems. We review the standard contraction theory on Euclidean spaces as well as little-known results for Riemannian manifolds. Special emphasis is placed on the setting of non-Euclidean norms and the recently introduced weak pairings for the ℓ1 and ℓ∞ norms. We highlight recent results on explicit and implicit fixed point schemes for non-Euclidean contracting systems.
Gyrofluid analysis of electron βe effects on collisionless reconnection

The linear and nonlinear evolutions of the tearing instability in a collisionless plasma with a strong guide field are analysed on the basis of a two-field Hamiltonian gyrofluid model. The model is valid for a low ion temperature and a finite $\beta _e$. The finite $\beta _e$ effect implies a magnetic perturbation along the guide field direction, and electron finite Larmor radius effects. A Hamiltonian derivation of the model is presented. A new dispersion relation of the tearing instability is derived for the case $\beta _e=0$ and tested against numerical simulations. For $\beta _e \ll 1$ the equilibrium electron temperature is seen to enhance the linear growth rate, whereas we observe a stabilizing role when electron finite Larmor radius effects become more relevant. In the nonlinear phase, stall phases and faster than exponential phases are observed, similarly to what occurs in the presence of ion finite Larmor radius effects. Energy transfers are analysed and the conservation laws associated with the Casimir invariants of the model are also discussed. Numerical simulations seem to indicate that finite $\beta _e$ effects do not produce qualitative modifications in the structures of the Lagrangian invariants associated with Casimirs of the model.
Cloud Failure Prediction with Hierarchical Temporal Memory: An Empirical Assessment

Hierarchical Temporal Memory (HTM) is an unsupervised learning algorithm inspired by the features of the neocortex that can be used to continuously process stream data and detect anomalies, without requiring a large amount of data for training nor requiring labeled data. HTM is also able to continuously learn from samples, providing a model that is always up-to-date with respect to observations.These characteristics make HTM particularly suitable for supporting online failure prediction in cloud systems, which are systems with a dynamically changing behavior that must be monitored to anticipate problems. This paper presents the first systematic study that assesses HTM in the context of failure prediction.The results that we obtained considering 72 configurations of HTM applied to 12 different types of faults introduced in the Clearwater cloud system show that HTM can help to predict failures with sufficient effectiveness (F-measure = 0.76), representing an interesting practical alternative to (semi-)supervised algorithms.
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback

First proposed by Seide et al. (2014) as a heuristic, error feedback (EF) is a very popular mechanism for enforcing convergence of distributed gradient-based optimization methods enhanced with communication compression strategies based on the application of contractive compression operators. However, existing theory of EF relies on very strong assumptions (e.g., bounded gradients), and provides pessimistic convergence rates (e.g., while the best known rate for EF in the smooth nonconvex regime, and when full gradients are compressed, is O(1/T ), the rate of gradient descent in the same regime is O(1/T )). Recently, Richtárik et al. (2021) (2021) proposed a new error feedback mechanism, EF21, based on the construction of a Markov compressor induced by a contractive compressor. EF21 removes the aforementioned theoretical deficiencies of EF and at the same time works better in practice. In this work we propose six practical extensions of EF21, all supported by strong convergence theory: partial participation, stochastic approximation, variance reduction, proximal setting, momentum and bidirectional compression. Several of these techniques were never analyzed in conjunction with EF before, and in cases where they were (e.g., bidirectional compression), our rates are vastly superior.
Burst Image Restoration and Enhancement

Modern handheld devices can acquire burst image sequence in a quick succession. However, the individual acquired frames suffer from multiple degradations and are misaligned due to camera shake and object motions. The goal of Burst Image Restoration is to effectively combine complimentary cues across multiple burst frames to generate high-quality outputs. Towards this goal, we develop a novel approach by solely focusing on the effective information exchange between burst frames, such that the degradations get filtered out while the actual scene details are preserved and enhanced. Our central idea is to create a set of pseudo-burst features that combine complimentary information from all the input burst frames to seamlessly exchange information. However, the pseudo-burst cannot be successfully created unless the individual burst frames are properly aligned to discount inter-frame movements. Therefore, our approach initially extracts pre-processed features from each burst frame and matches them using an edge-boosting burst alignment module. The pseudo-burst features are then created and enriched using multi-scale contextual information. Our final step is to adaptively aggregate information from the pseudo-burst features to progressively increase resolution in multiple stages while merging the pseudo-burst features. In comparison to existing works that usually follow a late fusion scheme with single-stage upsampling, our approach performs favorably, delivering state-of-the-art performance on burst super-resolution, burst low-light image enhancement and burst denoising tasks. The source code and pre-trained models are available at https://github.com/akshaydudhane16/BIPNet.
Optimal (Euclidean) Metric Compression

We study the problem of representing all distances between $n$ points in $\mathbb R^d$, with arbitrarily small distortion, using as few bits as possible. We give asymptotically tight bounds for this problem, for Euclidean metrics, for $\ell_1$ (a.k.a.~Manhattan) metrics, and for general metrics. Our bounds for Euclidean metrics mark the first improvement over compression schemes based on discretizing the classical dimensionality reduction theorem of Johnson and Lindenstrauss (Contemp.~Math.~1984). Since it is known that no better dimension reduction is possible, our results establish that Euclidean metric compression is possible beyond dimension reduction.
Efficient Sharpness-aware Minimization for Improved Training of Neural Networks

Overparametrized Deep Neural Networks (DNNs) often achieve astounding performances, but may potentially result in severe generalization error. Recently, the relation between the sharpness of the loss landscape and the generalization error has been established by Foret et al. (2020), in which the Sharpness Aware Minimizer (SAM) was proposed to mitigate the degradation of the generalization. Unfortunately, SAM s computational cost is roughly double that of base optimizers, such as Stochastic Gradient Descent (SGD). This paper thus proposes Efficient Sharpness Aware Minimizer (ESAM), which boosts SAM s efficiency at no cost to its generalization performance. ESAM includes two novel and efficient training strategies-StochasticWeight Perturbation and Sharpness-Sensitive Data Selection. In the former, the sharpness measure is approximated by perturbing a stochastically chosen set of weights in each iteration; in the latter, the SAM loss is optimized using only a judiciously selected subset of data that is sensitive to the sharpness. We provide theoretical explanations as to why these strategies perform well. We also show, via extensive experiments on the CIFAR and ImageNet datasets, that ESAM enhances the efficiency over SAM from requiring 100% extra computations to 40% vis-a-vis base optimizers, while test accuracies are preserved or even improved.
DeepBBS: Deep Best Buddies for Point Cloud Registration

Recently, several deep learning approaches have been proposed for point cloud registration. These methods train a network to generate a representation that helps finding matching points in two 3D point clouds. Finding good matches allows them to calculate the transformation between the point clouds accurately. Two challenges of these techniques are dealing with occlusions and generalizing to objects of classes unseen during training. This work proposes DeepBBS, a novel method for learning a representation that takes into account the best buddy distance between points during training. Best Buddies (i.e., mutual nearest neighbors) are pairs of points nearest to each other. The Best Buddies criterion is a strong indication for correct matches that, in turn, leads to accurate registration. Our experiments show improved performance compared to previous methods. In particular, our learned representation leads to an accurate registration for partial shapes and in unseen categories. Our code is publicly available 1
Three Operator Splitting with Subgradients, Stochastic Gradients, and Adaptive Learning Rates

Three Operator Splitting (TOS) (Davis&Yin, 2017) can minimize the sum of multiple convex functions effectively when an efficient gradient oracle or proximal operator is available for each term. This requirement often fails in machine learning applications: (i) instead of full gradients only stochastic gradients may be available; and (ii) instead of proximal operators, using subgradients to handle complex penalty functions may be more efficient and realistic. Motivated by these concerns, we analyze three potentially valuable extensions of TOS. The first two permit using subgradients and stochastic gradients, and are shown to ensure a $\mathcal{O}(1/\sqrt{t})$ convergence rate. The third extension AdapTOS endows TOS with adaptive step-sizes. For the important setting of optimizing a convex loss over the intersection of convex sets AdapTOS attains universal convergence rates, i.e., the rate adapts to the unknown smoothness degree of the objective. We compare our proposed methods with competing methods on various applications.
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees

Variational inequalities in general and saddle point problems in particular are increasingly relevant in machine learning applications, including adversarial learning, GANs, transport and robust optimization. With increasing data and problem sizes necessary to train high performing models across various applications, we need to rely on parallel and distributed computing. However, in distributed training, communication among the compute nodes is a key bottleneck during training, and this problem is exacerbated for high dimensional and over-parameterized models. Due to these considerations, it is important to equip existing methods with strategies that would allow to reduce the volume of transmitted information during training while obtaining a model of comparable quality. In this paper, we present the first theoretically grounded distributed methods for solving variational inequalities and saddle point problems using compressed communication: MASHA1 and MASHA2. Our theory and methods allow for the use of both unbiased (such as Rand$k$; MASHA1) and contractive (such as Top$k$; MASHA2) compressors. New algorithms support bidirectional compressions, and also can be modified for stochastic setting with batches and for federated learning with partial participation of clients. We empirically validated our conclusions using two experimental setups: a standard bilinear min-max problem, and large-scale distributed adversarial training of transformers.
Cut the CARP: Fishing for zero-shot story evaluation

Recent advances in large-scale language models (Raffel et al., 2019; Brown et al., 2020) have brought significant qualitative and quantitative improvements in machine-driven text generation. Despite this, generation and evaluation of machine-generated narrative text remains a challenging problem. Objective evaluation of computationally-generated stories may be prohibitively expensive, require meticulously annotated datasets, or may not adequately measure the logical coherence of a generated story's narratological structure. Informed by recent advances in contrastive learning (Radford et al., 2021), we present Contrastive Authoring and Reviewing Pairing (CARP): a scalable, efficient method for performing qualitatively superior, zero-shot evaluation of stories. We show a strong correlation between human evaluation of stories and those of CARP. Model outputs more significantly correlate with corresponding human input than those language-model based methods which utilize finetuning or prompt engineering approaches. We also present and analyze the Story-Critique Dataset, a new corpora composed of 1.3 million aligned story-critique pairs derived from over 80,000 stories. We expect this corpus to be of interest to NLP researchers.
Scene Transformer: A unified architecture for predicting multiple agent trajectories

Predicting the motion of multiple agents is necessary for planning in dynamic environments. This task is challenging for autonomous driving since agents (e.g. vehicles and pedestrians) and their associated behaviors may be diverse and influence one another. Most prior work have focused on predicting independent futures for each agent based on all past motion, and planning against these independent predictions. However, planning against independent predictions can make it challenging to represent the future interaction possibilities between different agents, leading to sub-optimal planning. In this work, we formulate a model for predicting the behavior of all agents jointly, producing consistent futures that account for interactions between agents. Inspired by recent language modeling approaches, we use a masking strategy as the query to our model, enabling one to invoke a single model to predict agent behavior in many ways, such as potentially conditioned on the goal or full future trajectory of the autonomous vehicle or the behavior of other agents in the environment. Our model architecture employs attention to combine features across road elements, agent interactions, and time steps. We evaluate our approach on autonomous driving datasets for both marginal and joint motion prediction, and achieve state of the art performance across two popular datasets. Through combining a scene-centric approach, agent permutation equivariant model, and a sequence masking strategy, we show that our model can unify a variety of motion prediction tasks from joint motion predictions to conditioned prediction.
RotoGrad: Gradient Homogenization in Multitask Learning

Multitask learning is being increasingly adopted in applications domains like computer vision and reinforcement learning. However, optimally exploiting its advantages remains a major challenge due to the effect of negative transfer. Previous works have tracked down this issue to the disparities in gradient magnitudes and directions across tasks, when optimizing the shared network parameters. While recent work has acknowledged that negative transfer is a two-fold problem, existing approaches fall short as they only focus on either homogenizing the gradient magnitude across tasks; or greedily change the gradient directions, overlooking future conflicts. In this work, we introduce RotoGrad, an algorithm that tackles negative transfer as a whole: it jointly homogenizes gradient magnitudes and directions, while ensuring training convergence. We show that RotoGrad outperforms competing methods in complex problems, including multi-label classification in CelebA and computer vision tasks in the NYUv2 dataset. A Pytorch implementation can be found in https://github.com/adrianjav/rotograd.
Expected signature of stopped Brownian motion on $d$-dimensional $C^{2, \alpha}$-domains has finite radius of convergence everywhere: $2\leq d \leq 8$

A fundamental question in rough path theory is whether the expected signature of a geometric rough path completely determines the law of signature. One sufficient condition is that the expected signature has infinite radius of convergence, which is satisfied by various stochastic processes on a fixed time interval, including the Brownian motion. In contrast, for the Brownian motion stopped upon the first exit time from a bounded domain Ω, it is only known that the radius of convergence for the expected signature on sufficiently regular Ω is strictly positive everywhere, and that the radius of convergence is finite at some point when Ω is the 2-dimensional unit disc ([1]). In this paper, we prove that on any bounded C-domain Ω ⊂ R with 2 ≤ d ≤ 8, the expected signature of the stopped Brownian motion has finite radius of convergence everywhere. A key ingredient of our proof is the introduction of a “domain-averaging hyperbolic development” (see Definition 4.1), which allows us to symmetrize the PDE system for the hyperbolic development of expected signature by averaging over rotated domains.
A Logic-Based Framework for Natural Language Inference in Dutch

We present a framework for deriving inference relations between Dutch sentence pairs. The proposed framework relies on logic-based reasoning to produce inspectable proofs leading up to inference labels; its judgements are therefore transparent and formally verifiable. At its core, the system is powered by two λ-calculi, used as syntactic and semantic theories, respectively. Sentences are first converted to syntactic proofs and terms of the linear λ-calculus using a choice of two parsers: an Alpino-based pipeline, or Neural Proof Nets. The syntactic terms are then converted to semantic terms of the simply typed λ-calculus, via a set of hand designed typeand termlevel transformations. Pairs of semantic terms are then fed to an automated theorem prover for natural logic which reasons with them while using the lexical relations found in the Open Dutch WordNet. We evaluate the reasoning pipeline on the recently created Dutch natural language inference dataset, and achieve promising results, remaining only within a 1.1–3.2% performance margin to strong neural baselines. To the best of our knowledge, the reasoning pipeline is the first logic-based system for Dutch. The code is available at git.io/JzdGd.
Biomechanical Pulping of Poplar with Crude Enzyme Secreted from Trametes sp. lg-9

Crude enzyme secreted from Trametes sp. lg-9 was applied in bio-mechanical pulping of poplar. The Canadian Standard Freeness (CSF) of the pulp pretreated with crude enzyme by a charge of 8 IU g-1 pulp was 215 mL. However, the CSF of untreated pulp was 235 mL at the same refining revolutions of 15000. Also, the energy consumption during refining was significantly lowered. The brightness of enzyme-pretreated pulp was increased by 2%, and the light absorption coefficient and opacity were decreased slightly. The effect of H2O2 (P) and CH3COOOH-H2O2 (PaP) bleaching were reinforced, and the brightness of pulp was further enhanced when the dosage of crude enzyme was great than 8 IU g-1 pulp. However, the content of fines was decreased, and the lowest value was 6.99% when the dosage of crude enzyme was 8 IU g-1 pulp. The results of this work will be valuable for future possible commercialization.